In designing OSMPS we have attempted to keep the number of dependencies to a minimum. However, there are a few requirements. Each package listed below is necessary to allow OSMPS to run its calculations and process data. It uses a python based user interface, and compiles and runs calculations through a Fortran library. The following dependencies are mandatory:
- Python 2.6 or 2.7 (Python 3.x was successfully tested for v2).
- NumPy, a library containing numerical routines for Python.
- SciPy, a library containing additional scientific routines for Python.
- A Fortran 95 compiler. OSMPS has been extensively tested for compatibility with gfortran, g95, and ifort.
- BLAS LAPACK libraries.
- ARPACK. It is recommended to install this package from arpack-ng rather than from the Rice webpage.
Further dependencies are necessary when using the built-in plotting in python. These optional packages are:
- Matplotlib for python
- Converter dvipng
The following installation instructions detail the latest stable release of the software. We can not guarantee that more recent releases of dependencies will work with OSMPS and additional configuration or integration between dependencies and the OSMPS software might be necessary.
The general installation procedure should follow the order of the dependencies as listed above. Before installing any of the packages, check if your system already has them. For example, Mac OS has preinstalled Fortran compilers and most updated Linux-based OSes rely on python. Installation instructions are provided in the following subsections for different operating systems.
Ideally, LAPACK and BLAS libraries tuned to the specific architecture should be used, such as ATLAS or MKL. GOTOBLAS should not be used, as it has bugs in LAPACK routines used by OSMPS and is no longer under development. Because it is open source, relatively efficient, and overall effective, we recommend using ATLAS. To tune these libraries, see the information to install and run ATLAS, described in Section Installing ATLAS.
Installing dependent packages on Linux¶
Here we describe the installation procedure assuming a new Ubuntu installation. We have tested this procedure on Ubuntu 12 and Ubuntu 14. The second part of this section contains general instructions for unix OS or Ubuntu which has already been used. A detailed explanation of the code below is provided after the last line of code. The following commands can be entered directly at the command line or placed in a script. Executing those lines will cover all dependencies to run all the example files:
osmps@manual:~$ sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) main universe" osmps@manual:~$ sudo apt-get update osmps@manual:~$ sudo apt-get install python-numpy osmps@manual:~$ sudo apt-get install python-scipy osmps@manual:~$ sudo apt-get install python-matplotlib # optional package for built-in plotting osmps@manual:~$ sudo apt-get install dvipng # optional package for built-in plotting osmps@manual:~$ sudo apt-get install gfortran osmps@manual:~$ sudo apt-get install libarpack2 osmps@manual:~$ sudo apt-get install texlive # optional package for built-in plotting osmps@manual:~$ sudo apt-get install texlive-latex-extra # optional package for built-in plotting
The packages in Ubuntu are contained in different repositories. In addition to the repository main we need packages from the repository universe, specifically scipy. The first two lines add the universe repository and update the configuration.
Both Ubuntu versions tested (12, 14) come with python2.7 installed by default. Therefore, we can immediately start to install numpy and scipy. During the installation of numpy and scipy, some libgfortran3-packages are already installed. If you want to use python's built-in plotting capabilities, you must execute the lines for the packages matplotlib and dvipng.
During the installation of the OSMPS, the fortran files of the OSMPS library are compiled. Therefore, you need to install the fortran compiler gfortran itself. Further arpack is necessary.
The last two lines containing texlive-packages are again optional and only referenced when using the built-in plotting of python.
If you do not have a brand new Ubuntu installation (or other unix distributions), you can follow the steps below to see which dependencies are already installed instead of executing the lines above. The default version of python can be found by typing either one of the following lines:
osmps@manual:~$ python --version osmps@manual:~$ python
The first line will print the version on the screen. In contrast the second opens a python-shell where in the header the version is shown as well. You can leave the python-shell by entering quit() or exit(). If the version is not 2.6 or 2.7, check with
osmps@manual:~$ whereis python
if there are other versions of python available. If version 2.7 (2.6) are available on your computer, but not the default, you can replace all calls to python with calls to python2.7 (python2.6) without changing your default version of python. The commands are then executed with version 2.7 (2.6) instead of the default version. We point out again, that python 3.x uses different syntax and was tested successfully with release v2. However, if you encounter unresolved problems using python 3.x, please report them to us in the forum.
Once you know that python is available, you must check if the modules numpy, scipy and matplotlib are installed. matplotlib only applies when you want to plot. Any time you are in the terminal, you can enter a Python shell by simply typing python, and inputs will be interpreted through Python syntax. The following python inputs will raise an ImportError if the modules are not available.
>> import numpy as np >> import scipy >> import matplotlib
If you have ImportErrors for numpy and/or scipy, you will need to install these packages. The same applies for the optional package matplotlib if you want to plot within python.
Alternatively, you can save all packages installed on your system into a file. This can be done through the command:
osmps@manual:~$ dpkg --get-selections | grep -v deinstall > ~/SaveDIRECTORY/FileNAME.txt
This will save a list of all installed packaged to a text file by the name FileNAME.txt. This file is located in the folder or file par SaveDIRECTORY. (The directory or path SaveDIRECTORY must be present before executing this command and existing files FileNAME.txt in this directory will be overwritten.) When opening FileNAME, you should see the following packages when you have properly installed everything: libpython2.6 or libpython2.7, python-numpy, python-scipy in regard to the python installation. These python-dependencies should include automatically the packages libfortran3, liblapack3 and libblas3. Furthermore check for gfortran and libarpack2 and install them if necessary. For LAPACK and ARPACK, there may be an updated version available, but the previously mentioned packages have been tested and shown to work seamlessly with OSMPS. For the optional packages look for python-matplotlib, dvipng, texlive and texlive-latex-extra and install them if they are not yet available.
If some or all of these packages are not installed, you must install them. The easiest way to install most of the packages on Linux is directly through the terminal using a front-end packaging tool called apt-get, which is installed by default.
An alternative solution is aptitude, which can be installed through the Software Center in Ubuntu distros or its equivalent in other distros if it is not available (check availability with info aptitude). The command
osmps@manual:~$ sudo apt-get install aptitude
installs aptitude via the terminal. In order to search for packages use aptitude search package-name and install it then with sudo aptitude install package-name. Further useful features are: a mutt-like syntax for matching packages in a flexible manner, dselect-like persistence of user actions, the ability to retrieve and display the Debian changelog of most packages, and a command-line mode similar to that of apt-get. aptitude is also y2K-compliant, non-fattening, naturally cleansing, and house broken. It should be noted that aptitude and apt-get will only work on Unix and Debian based OSes.
Depending on your operating system, you may have to use similar package managers like yum or yumex as available on systems like Fedora or Redhat. The library names should be the same, the command should be replaced with the appropriate package retriever. In order to get all the missing dependencies, you can execute the corresponding sudo apt-get install command presented at the beginning of this section.
Installing dependent packages on Mac OSX¶
Most of the required packages can be installed using MacPorts. If you do not yet have MacPorts installed, you may install it by following the directions found at MacPortsInstall. Once MacPorts is installed, the g95 compiler can be installed using:
osmps@manual:~$ sudo port install g95
Python and the associated libraries can be installed with:
osmps@manual:~$ sudo port install py27-numpy py27-scipy python27 arpack osmps@manual:~$ sudo port select --set python python27
Note that MacPorts puts libraries in /opt/local/lib/ by default, and so these libraries may not be found when attempting to link, e.g. Arpack, as -larpack. You can either copy all of the libarpack files from /opt/local/lib/ to /usr/local/lib/, like so:
osmps@manual:~$ sudo cp /opt/local/lib/libarpack* /usr/local/lib/
or you may specify the path to each library explicitly using the
TemplateGenerator.Writemakefile() function where the options
It may occur that the newly installed version of Python is trying to link to the old Python libraries. This has appeared as an issue in importing urandom. One resolution of the issue is to call hash -r in the terminal, see on this posting.
Macintosh already has ATLAS, a high-performance BLAS and LAPACK implementation, as part of its veclib framework. Instructions to build using these libraries are given in Sec. Building and installation.
Note once again that some example tutorial files require matplotlib as well as LaTeX with special fonts and a DVI to PNG converter, the necessary dependencies can be obtained by installing MACTEX and installing matplotlib like so,
osmps@manual:~$ sudo port install py27-matplotlib
Installing dependent packages on Windows¶
On Windows, there are a number of bundle-installations of Python that include shells or GUIs, associated Python packages, and extended capabilities. There should be no issue using one of these extended bundles, unless it uses a version of Python that is not 2.6 or 2.7. However, here we present the method for installing Python from the original distributions from the original developers. We recommend using the Windows executable files, however you can use the tarfiles and follow a similar procedure outlined in Section Installing dependent packages on Linux.
First download the python installer which can be found here. Be sure to install the correct version of python, (either the 32-bit or the 64-bit) for you system. This information can be checked in the System information for you computer which can be accessed through the Control Panel or by simultaneously pressing the Windows button and Pause buttons on your keyboard.
Then download and install the numpy and scipy packages. Numpy and scipy can be downloaded directly from the unofficial Windows binaries provided from the website mentioned on the scipy's parent website. Otherwise, there is a number of distributions of python that one can install. When installing one of these distributions, you do not need to have python previously installed. The distribution will come with a local installation of python. Anaconda is the scientific python distribution used by some of the OSMPS alpha testers and has been shown to be compatible for proper installation.
To install a fortran compiler for Windows, we recommend Cygwin which is necessary to have installed before the linear algebra packages BLAS and LAPACK are installed and tuned with ATLAS. You can download and install the executable file for Cygwin. Run this executable file, setup.exe. ATLAS, which is used to tune the LAPACK and BLAS requires either Cygwin or MinGW.
Then you will need to install LAPACK and BLAS. This can be found at this website. ATLAS should then be installed. Refer to Section Installing ATLAS and to this discussion for specifics on how to install and run ATLAS for Windows.
ATLAS, or Automatically Tuned Linear Algebra Software, can be downloaded from this website. All documentation provided by the developers of ATLAS is available online. ATLAS creates a BLAS (Basic Linear Algebra Subprograms) library that will be optimized to your system upon installation. Furthermore, ATLAS will provide routines from LAPACK (Linear Algebra PACKage) that utilize Fortran for solving common numerical problems. ATLAS is not fully inclusive of the LAPACK library. Notably, ATLAS expedites LU, QR and Cholesky factorizations; it uses purely recursive solutions for LU and Cholesky routines, and a hybrid algorithm for QR factorization (more information here).
The first step to installing ATLAS is to obtain the tarfile, which can be downloaded from this webpage. You will also need the LAPACK library tarfile, which can be downloaded from the LAPACK website. The tarfile for LAPACK will be untarred during the installation process, but remember the file location for the tarfile. The ATLAS file should be open and extracted (or untarred) into the folder you want to install and keep ATLAS. If you are using a terminal or command prompt, you can do so through the following commands. Otherwise, use your preferred software to untar the file, and move the untarred contents into a given installation directory.
osmps@manual:~$ cd ~/InstallFOLDER osmps@manual:~$ bunzip2 -c ~/DownloadFOLDER/atlas3.10.0.tar.bz2 | tar xfm - osmps@manual:~$ mv ATLAS ATLAS3.10.0
Here we have a number of directories. ~/InstallFOLDER is the location you want ATLAS installed. Depending on you hard drive configuration, you may have a longer directory path than just a single folder, but the command cd ~/InstallFOLDER should bring the terminal to the installation directory. Then we have ~/DownloadFOLDER which is the location of tar file you previously downloaded. The file, atlas3.10.0.tar.bz.2, will be unzipped using the by decompressing (indicated from the -c flag) the file and will be extracted using the xfm file manager. Then the extracted folder named ATLAS should be moved into the InstallFOLDER and into a folder for the specific version, which we choose here as ATLAS3.10.0.
At this point, you should read through the errata file for common installation errors or problems at this point. If no errors persist, continue by turning off your CPU throttling. You may have to find instructions to turn off CPU throttling for you given system. For Linux machines, this can be done through the terminal with the following command:
osmps@manual:~$ /usr/bin/cpufreq-selector -g performance osmps@manual:~$ cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
The first line may work sufficiently for some systems. However, depending on your processor, and the number of cores you have, you may have to do some iteration of the second line, copying the system settings from cpu0 to cpu1 and so on and so forth for each core. For Mac OX, copy the folder named IOPlatformPluginFamily.kext which is in system folder /library/extensions, and put it on a thumbdrive or other external media. Move the entire folder named above to the trash and empty it. Then restart. You can drag the folder back to its location and restart when you want to reactivate speed step. For Windows machines you can turn off throttling in the Power Options Menu, under Custom Settings under the Option called Processor Power Management. The minimum processor state and maximum processor state should be set to 100% when plugged in. Note, that after you install ATLAS, you can restore CPU throttling.
The ATLAS installation should be done in five steps:
- Configure: Which designates the source directory (SRCdir) and a separate location (which may be a subfolder, but is not required) which acts as the build directory for the program (BLDdir).
- Build: This step automatically tunes ATLAS to your system, building the libraries such that they use the most efficient configuration for calculations.
- Check: After installation, ATLAS runs through a set of sanity tests verifying calculations with precoded solutions.
- Time: After installation, ATLAS re-times all of the kernels used, to verify that the installation uses the most efficient settings.
- Install: Completing all verifications, the installation is complete by copying the Build directory to a stable file location.
So assuming you are on a terminal based system, you should be able to enter the following commands to complete the full installation of ATLAS.
osmps@manual:~$ bunzip2 -c atlas3.10.x.tar.bz2 | tar xfm - # create SRCdir osmps@manual:~$ mv ATLAS ATLAS3.10.x # get unique dir name osmps@manual:~$ cd ATLAS3.10.x # enter SRCdir osmps@manual:~$ mkdir Linux_C2D64SSE3 # create BLDdir osmps@manual:~$ cd Linux_C2D64SSE3 # enter BLDdir osmps@manual:~$ ../configure -b 64 -D c -DPentiumCPS=2400 --prefix=/home/whaley/lib/atlas --with-netlib-lapack-tarfile=/home/whaley/dload/lapack-3.4.1.tgz
The above command should be all on a single line, and it moves back to the SRCdir and configures the installation to hardware specifics, indicating the install directory with prefix, an pointing to the tarfile download location.
osmps@manual:~$ make build # tune & build lib osmps@manual:~$ make check # sanity check correct answer osmps@manual:~$ make ptcheck # sanity check parallel osmps@manual:~$ make time # check if lib is fast osmps@manual:~$ make install # copy libs to install dir
Note that this set of commands includes untarring the LAPACK file, creating both the installation or source directory (SRCdir) and the build directory (BLDdir) where the files are installed, tested, and verified for optimal settings, and then transferred back to the SRCdir. For the long configure -b 64 -D c -DPentiumCPS=2400 command, there are a number of settings that you must match for your system. The flag -b indicates the bitwidth, and should appear as -b 32 or -b 64 depending on the bitwidth of your processor. The next portion of the command, -D c -DPentiumCPS=2400, should be custom to your machine. The command should follow -D c -DPentiumCPS=<your speed in MHz>. Alternatively, you can also use the cycle accurate wall timer instead of the processor timer by using the command -D c -DWALL. The latter command should be used if you are not on an x86 machine, or if your compiler is not gcc or icc.
Further installation instructions can be found in the third-party installation guide.
Building and installation¶
We will distinguish between two different build modes of OSMPS which we will call global installation and local installation. A global installation will allow for scripts using OSMPS to be called from anywhere, and a local installation requires that the Python libraries and Fortran executables exist in the calling directory. Global installation requires root access. If you are unsure about which kind of installation you want, do a local installation.
In case you want to switch between a local and global installation, the python
tools.runMPS() will use the local installation by default, if
the Fortran executable is available in the current folder. Otherwise python
assumes a global installation. An optional argument
RunDir='./') can be specified to use the global (local) installation
or any other path to a local installation.
Python is an interpreted language, and so the Python code comprising MPSPyLib does not need to be compiled. The Fortran code in MPSFortLib does need to be compiled, and a Python convenience script BuildOSMPS.py is included to facilitate this.
The OSMPS repository can be found on SourceForge. The full repository where all the source code can be downloaded can be found our SourceForge homepage. Further information can be found in Section Obtaining/Modifying the Source Code for obtaining the code.
To install OSMPS, we will focus on the files setup.py and BuildOSMPS.py. Whether you run these files from a command prompt or from a python GUI (generally dependent on your operating system), you will need to run both of these python files.
A global installation of the Python libraries adds them to the PYTHONPATH, which is a list of directory names where python goes to look for external code similar to the shell variable PATH. A global installation of MPSPyLib is achieved by calling the following
osmps@manual:~$ sudo python setup.py install
in the command line. If working on a Windows machine, or within a python GUI, you can open this file, setup.py, and then run the file within the GUI. In the main directory containing the MPSPyLib directory. One should be careful following a global installation that a different copy of the MPSPyLib directory does not exist in the calling directory, as any local copies of a module take precedence over globally installed modules.
We then call BuildOSMPS.py from the command line like so,
osmps@manual:~$ sudo python BuildOSMPS.py
where global installation and an Ubuntu operating system are default. Alternate
operating systems can be specified via the following command line options
as specified in
BuildOSMPS.main(). For example, to do a global
installation on a macintosh operating system one would submit the following
at the command line.
osmps@manual:~$ sudo python BuildOSMPS.py --os=mac
Running OSMPS code which has been globally installed may be accomplished via
tools.runMPS() function from MPSPyLib without the optional argument
RunDir as long there is no local installation in the same folder.
For the case with a local installation in the same folder, use
RunDir='' to use the global installation, as described previously.
In order to install the OSMPS locally, you need a copy (or a symbolic link) of the folders MPSFortLib and MPSPyLib and of the file BuildOSMPS.py to be present in your installation directory. (Symbolic links can be created with ln -s PATH_TO_TARGET LINK_NAME.) Then, in the command prompt, one calls
osmps@manual:~$ python BuildOSMPS.py --local='./'
to build the fortran executable in the current directory, assuming the default
settings for Ubuntu. To build the executable in a different directory,
--local='./' should instead be
To request the settings for an alternative operating system use the same
syntax as in the global installation, e.g.
--os=mac. If the local
path specified via
--local is different from the current directory,
the directory MPSPyLib must be available in the directory you have
specified if you plan to run simulations from said directory. If this is not
done, then any simulations you launch with the python front end will not be
able to import MPSPyLib.
Running OSMPS code which has been locally installed may be accomplished
tools.runMPS() function from MPSPyLib without the optional
RunDir, since the function always takes the local
installation first if available.
In order to delete the compiled fortran files, you can run the command
osmps@manual:~$ python BuildOSMPS.py --clean
When any other optional command line argument is invoked in the call, the script first cleans up and then compiles according to the given specifications.
The convenience function
BuildOSMPS.py contains arguments for the compile flags used
by MPSFortLib. The optional arguments are collected in the function
The convenience function
BuildOSMPS.py has a single optional argument
The convenience function
BuildOSMPS.py has a two optional arguments to control the
installation directory and the MPI installation (see function documentation).
Some common custom build operations are:
- To install in a place other than
/usr/local/bin/, pass the directory where MPSFortLib is to be installed as the optional argument
- To build the parallel libraries, modify the arguments of
TemplateGenerator.Writemakefile()collected in the function documentation to be appropriate for your parallel compiler, and then set the optional argument
- (Relevant only for developers) To compile and run in debug mode, specify
TemplateGenerator.Writemakefile(), and then call
Debug=True. This will run the simulation through valgrind to check for memory leaks.