I installed one of the gcc-toolset packages (gcc-toolset-9) available for RHEL 8 to attempt to access a more recent version of GLIBC and was able to download the tar.gz file and create a glibc-2.29/build folder. I have tried a few ways to link it but keep running into errors. Can anyone advise on code that can run in python to use the glibc-2.29 from the toolset for the necessary package? Code for installing a GCC toolset to download, build, and use glibc 2.29 locally follows:
! yum install gcc-toolset-9
import os
import subprocess
import sys
# Define paths
glibc_version = '2.29'
glibc_tarball_url = f'http://ftp.gnu.org/gnu/libc/glibc-{glibc_version}.tar.gz'
glibc_src_dir = f'glibc-{glibc_version}'
build_dir = f'{glibc_src_dir}/build'
install_dir = os.path.expanduser(f'~/glibc-{glibc_version}-install')
# Function to run shell commands
def run_command(command, cwd=None, check=True):
print(f"Running command: {command}")
result = subprocess.run(command, shell=True, cwd=cwd, check=check, text=True, capture_output=True)
print(result.stdout)
if result.stderr:
print(f"Error: {result.stderr}", file=sys.stderr)
return result
# Create a working directory
print(f"Creating working directory...")
run_command(f'mkdir -p ~/glibc-{glibc_version}', check=False)
Upgrading to GLIBC 2.29 looks like the obvious choice here but is not the solution here. GLIBC is so deeply entrenched into an OS that upgrading the same without upgrading the whole OS does typically not end very well...
The main culprit here is fairgbm whose available python wheel has been compiled against GLIBC 2.29. Python wheels are great because they already contain binaries and hence you do not need to recompile from source. But if they are compiled against a too recent GLIBC you're stuck.
You however can force pip to NOT use an existing wheel for certain packages and instead build a wheel itself. In the case of fairgbm / aequitas you can do this via
pip install --no-binary fairgbm aequitas
If you use the approach above I think you no longer need any gcc-toolset either (you however can go up to gcc-toolset-13 (at least this is available in RockyLinux 8).
PS: You will ensure to install cmake to make the above work. fairgbm uses cmake for building the software.
Thank you, Michael. This is helpful insight. Are you able to help me figure out where to integrate your pip install recommendation into my current code?:
from aequitas import Audit
import pandas as pd
import seaborn as sns
from aequitas.audit import Audit
import aequitas.plot as ap
import os
import pyodbc
import matplotlib.pyplot as plt
import plotly.express as px
from aequitas.group import Group
from aequitas.bias import Bias
from aequitas.fairness import Fairness
from aequitas.plotting import Plot
from aequitas import plot
from aequitas.flow.utils.logging import clean_handlers
from aequitas.flow.utils.colab import get_examples
version('fairGBM')
from aequitas.flow import Experiment
from aequitas.flow.methods.postprocessing import BalancedGroupThreshold
I am not sure how you installed aequitas in the first place but the easiest IMHO would be to remove aequitas and start the installation with the command provided (maybe create a new virtual env). Alternatively you coud run in your jupyter notebook
which would force reinstall of fairgbm + dependencies. There is a small chance that it may mess up some dependencies for aequitas so the "start fresh" approach feels better to me.
Thank Michael! I am working on this project as well and running into an error. It says I need to install Cmake and dependencies - I have done this but still getting the same thing.
After trying to install it this way, I am getting the following error:
My gut is telling me this is a path issue, but I don't know what to do about it
Which OS are you working under ? RHEL 8 as well ?
NAME="Red Hat Enterprise Linux"
VERSION="8.7 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.7"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.7 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="Red Hat Enterprise Linux | Red Hat Product Documentation"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.7
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.7"
How did you install cmake ?
Pip install cmake
I have also tried to use ! Yum install - y cmake but need admin privileges
What does cmake --version report on your system ?
It can't find it
Do you find any additional errors or information in /hhc_home/lees149/FairGBM_compilation.log ?
The log looks empty?
I am guessing that you installed cmake via pip outside of any virtual environment, right ? In this case you will need to add the directory (presumably /hhc_home/lees149/.local/bin) to your PATH environment variable. e.g. run
export PATH=/hhc_home/lees149/.local/bin:$PATH
and see if cmake can be found then.
While this may actually work, I would recommend to use a dedicated python venv for your fairGBM/aequitas work. More specifically I would suggest to create a project folder for your aequitas work, and within that folder run
The above commands should work . Now, every time you wanrt to work with aequitas you simply need to go back into the project folder, run source .venv/bin/activate and you have all your aequitas/fairGBM stuff available again. Of course you also can register the new virtual environment in jupyter/jupyterlab for better integration - I will leave that up to you as an exercise.
The benefits of using virtual envs are that you really can make them project specific and avoid overwriting or inadvertently updating a package like you would if you would install everything in one user default library path.
