S3FS: Mounting Amazon S3 Buckets on CentOS & Fedora Systems

    By 4 Mins Read

    Amazon S3 is a highly scalable and durable object storage service that allows you to store and retrieve any amount of data from anywhere on the web. S3FS is a FUSE (Filesystem in Userspace) interface for Amazon S3 that allows you to mount Amazon S3 buckets as a local file system on your CentOS or Fedora system. This allows you to easily access and manipulate your S3 data as if it were stored on a local file system. In this article, we’ll show you how to mount an Amazon S3 bucket on a CentOS or Fedora system using S3FS.

    Advertisement

    Before we begin, make sure that you have the following:

    1. An Amazon S3 account with at least one bucket created
    2. An access key and secret key with access to your S3 bucket
    3. A CentOS or Fedora system with root access
    4. S3FS installed on your system

    Here’s the step-by-step guide to mounting an Amazon S3 bucket on a CentOS or Fedora system using S3FS:

    Step 1: Remove Existing Packages

    First, check if you have any existing s3fs or fuse packages installed on your system. If installed already remove it to avoid any file conflicts.

    yum remove fuse fuse-s3fs

    Step 2: Install Required Packages

    After removing packages. First, we will install all the dependencies for fuse and s3cmd. Install the required packages to the system using the following command.

    yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap

    Step 3: Download and Compile Fuse

    Download and compile latest version of fuse source code. For this article, we are using fuse version 3.13. Following the set of commands will compile the fuse and add the fuse module to the kernel.

    1. Downlaod fuse source code:
      cd /usr/src/ 
wget https://github.com/libfuse/libfuse/releases/download/fuse-3.13.0/fuse-3.13.0.tar.xz
    2. Extract the archive file
      tar Jxf fuse-3.13.0.tar.xz 
cd fuse-3.13.0
    3. Configure the source code and compile it.
      ./configure --prefix=/usr/local 
make && make install
    4. Set the environment variable and enable the fuse module.
      export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig 
ldconfig 
modprobe fuse

    Step 4: Download and Compile Latest S3FS

    Download and compile latest version of s3fs source code. For this article, we are using s3fs version 1.74. After downloading extract the archive and compile the source code in the system.

    1. Download the source code from the git repository.
      cd /usr/src/ 
git clone https://github.com/s3fs-fuse/s3fs-fuse.git
    2. Configure the source code
      cd s3fs-fuse 
./autogen.sh 
./configure
    3. Compile and install s3fs
      make && make install

    Step 5: Create Credentials File

    Also In order to configure s3fs, we would require an Access Key and Secret Key of your S3 Amazon account. Get these security keys from Here.

    Create a password file to store your access and secret keys:

    echo AWS_ACCESS_KEY:AWS_SECRET_KEY > ~/.passwd-s3fs 
chmod 600 ~/.passwd-s3fs

    Note: Change AWS_ACCESS_KEY and AWS_SECRET_KEY with your actual Amazon S3 access and secret keys. This file stores your access and secret keys securely, and the permissions are set to prevent unauthorized access.

    Step 6: Mount S3 Bucket

    Create a directory to mount the S3 bucket:

    mkdir /tmp/cache /s3-mount

    This creates a directory where the S3 bucket will be mounted. You can choose a different directory name as desired.

    Modify the permissions for the mount directory:

    chmod 777 /tmp/cache /s3-mount

    This sets the permissions for the mount directory to allow read/write access for all users.

    s3fs -o use_cache=/tmp/cache BUCKET_NAME /s3-mount

    Replace BUCKET_NAME with the actual name of your Amazon S3 bucket. This command mounts the S3 bucket at the specified mount directory.

    Step 7: Verify Mount

    The df command displays the disk usage and mounted file systems on your system. If the S3 bucket is successfully mounted, you should see the mount directory listed with its corresponding S3 bucket name.

    df -h

    That’s it! You have now successfully mounted an Amazon S3 bucket on your CentOS or Fedora system using S3FS. You can now access and manipulate the data in your S3 bucket as if it were stored on a local file system.

    Conclusion

    In conclusion, mounting an Amazon S3 bucket on a CentOS or Fedora system using S3FS is a simple and effective way to access your S3 data. With S3FS, you can easily access and manipulate your S3 data as if it were stored on a local file system, making it easier to work with your S3 data in your CentOS or Fedora environment.

    Share.

    Related Posts

    View 39 Comments

    39 Comments

    2. donnyO on

      any idea how to mount local S3 (not on AWS) created on PURE storage. I got the bucket name, Bucket Endpoint, Bucket Access Key and bucket Secret Access Key .

      Reply
    4. Deepak Kumar on

      Iā€™m got error :

      s3fs: HTTP: 403 Forbidden ā€“ it is likely that your credentials are invalid

      are you have solved ??

      Reply
    7. Arkadiusz on

      On Ubuntu steps 1-4 can be replaced with one command:
      sudo apt-get install s3fs
      Do this instead of steps 1-4 and continue from step 5. It works – tested.

      Reply
    9. Arunkumar on

      I am getting below error.

      fuse: warning: library too old, some operations may not not work

      # rpm -qa | grep fuse
      libconfuse-2.7-4.el6.x86_64
      fuse-libs-2.8.3-5.el6.x86_64
      #

      though latest package is not available in Yum repo. please suggest

      Reply
    10. Khan on

      Hello, i followed your guide (on Ubuntu 14.04, Bitnami – EC2 ) , i am getting following error at “make”
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `ldap_nex[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
      collect2: error: ld returned 1 exit status
      make[2]: *** [s3fs] Error 1
      make[2]: Leaving directory `/tmp/s3fs-fuse-1.80/src’
      make[1]: *** [all-recursive] Error 1
      make[1]: Leaving directory `/tmp/s3fs-fuse-1.80′
      make: *** [all] Error 2

      Reply
    11. Abu Kamal on

      use this if you get “bash: s3fs: command not found”
      sudo /usr/local/bin/s3fs -o use_cache=/tmp/cache mydbbackup /s3mnt

      Reply
    12. Tom Fazakas on

      Those of you having an error on CentOS: s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

      yum install fuse-libs

      Reply
    13. James on

      Great article! Worked a treat….

      only issue was my library path required updating as when I ran s3fs it couldn’t find the fuse dependancies.

      LD_LIBRARY_PATH=/usr/local/lib
      export LD_LIBRARY_PATH

      That did the trick šŸ™‚

      Reply
    14. Bhavesh on

      Hi..
      i have mounted s3 sucessfully , but when i try to “cd ” to mounted dirs it says “operation not permitted”

      cd: app_logs/: Operation not permitted

      below is permission for dir

      d———. 1 root root 1 May 6 2015 app_logs

      below command used
      s3fs -o use_cache=/tmp/cache s3bukcket /s3mnt

      what permission i need to set for bucket or bucket folder ?
      am i writing wrong command for mount?

      Reply
    15. Mehul on

      Hi Rahul,

      Can you please share the steps to mount Amazon S3 bucket on windows OS for both 2008 & 2012?

      Thanks & Regards,
      Mehul

      Reply
    17. David on

      Hi all, I’m receiving the following error after trying to mount the bucker:

      # s3fs -o user_cache=/tmp/cache lehar-backup /s3mnt
      s3fs: /lib/libfuse.so.2: version `FUSE_2.8′ not found (required by s3fs)

      I downloaded fuse 2.8 and compiled it per the instructions however if there is something I’m missing I’d love to know what that is. Let me know what information you might need. Thanks for any help you can provide.

      ~Dave

      Reply
    18. Srikanth on

      My disk is full after mount and not able to use the system. Ideally it should not take disk space.
      How can i resolve the issue?

      Filesystem 1K-blocks Used Available Use% Mounted on
      /dev/xvda1 10321208 10297608 0 100% /
      none 847476 0 847476 0% /dev/shm
      s3fs 274877906944 0 274877906944 0% /s3mnt

      Reply
    23. Vineshkumar on

      Hi Rahul,

      Thanks for the tutorial. All worked fine, except this messsage – Transport endpoint is not connected. Unmounted and mounted again using the link you shared in comments. I am using s3fs to connect to Google Cloud Storage.

      Kindly share some pointers or might be issue with the permission level mentioned above.

      Reply
    24. Bruno on

      Willian

      do it:

      # vim /etc/ld.so.conf

      /usr/local/lib
      include ld.so.conf.d/*.conf

      # ldconfig
      # s3fs -o use_cache=/tmp/cache ****.******.*** /s3mnt

      Reply
    26. William on

      I have a problem, can you helpme?

      [[email protected]*********** s3fs-1.74]# s3fs -o use_cache=/tmp/cache ****.******.*** /s3mnt
      s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

      Reply
        • bo on

          The problem was permissions on S3 bucket. Maybe you could add some recommendations on this subject? It would make your article more complete.

          Thank you

          Reply
          • Rahul on

            Hi bo,

            We would be thankful for your contributions and make the article more useful for users.

            We will also check for the permission problems.

            Reply
            • bo on

              I would also add info on automatic mounting the s3 bucket. Something like editing /etc/fdtab to add
              s3fs#bucketname mountpath fuse allow_other,_netdev,nosuid,nodev,url=https://s3.amazonaws.com 0 0

              Reply
            • Anil A on

              I have followed the tutorial and get to the state of “Transport endpoint is not connected”. Any updates on what permissions need to be set on S3 bucket?

              Reply

    Leave A Reply