Modular Installation of Calculator Packages in HPC Environment
2024-11-02
To record the installation of Python packages for high-throughput screening on a certain HPC platform and the modular process.
1. Preparation work
# Create necessary directories
mkdir -p /work/home/username/apprepo/camkit # Software Installation Directory
mkdir -p /work/home/username/apprepo/modules # Module File Directory
2. Install Python packages
# Load necessary modules
module load compiler/intel/2021.3.0
module load python/3.8.10
# Install Python packages (using AliCloud mirror sources for acceleration)
cd /path/to/package
python -m pip install --prefix=/work/home/username/apprepo/camkit -i https://mirrors.aliyun.com/pypi/simple/ .
3. Create module file
# Create module file
cat > /work/home/username/apprepo/modules/camkit-1.0 << 'EOF'
#%Module1.0
proc ModulesHelp { } {
puts stderr "This module adds CAMkit to your environment"
}
module-whatis "CAMkit Python package"
# Load dependent modules
(depending on the currently available modules; if they do not exist, you can create these required modules first or use conda)
module load compiler/intel/2021.3.0
module load python/3.8.10
# Set environment variables
set CAMKIT_ROOT /work/home/username/apprepo/camkit
# Add Python path
prepend-path PYTHONPATH $CAMKIT_ROOT/lib/python3.8/site-packages
# Add executable files to PATH
prepend-path PATH $CAMKIT_ROOT/bin
# Set other necessary environment variables
setenv CAMKIT_HOME $CAMKIT_ROOT
EOF
# Set module file permissions
chmod 644 /work/home/username/apprepo/modules/camkit-1.0
4. Configure module path
# Add module path to environment variable (If sharing is needed, it need be added to the public path)
echo 'export MODULEPATH=/work/home/username/apprepo/modules:$MODULEPATH' >> ~/.bashrc
source ~/.bashrc
5. Verify Installation
# Check if the module is visible
module avail camkit-1.0
# Load Module
module load camkit-1.0
# Test Python Package
python -c "import camkit; print(camkit.__file__)"
If return: "xxxx/camkit/lib/python3.8/site-packages/camkit/__init__.py", then succeed.
Note: Module files are usually set to 644 permissions to ensure that other users have appropriate access rights.
Common Commands
# View Available Modules
module avail
# Load Module
module load camkit-1.0
# Unload Module
module unload camkit-1.0
# View module details
module show camkit-1.0