• Home
  • Ubuntu 18.04
    • Whats New?
    • Upgrade Ubuntu
    • Install Java
    • Install Node.js
    • Install Docker
    • Install Git
    • Install LAMP Stack
  • Tutorials
    • AWS
    • Shell Scripting
    • Docker
    • Git
    • MongoDB
  • Funny Tools
  • FeedBack
  • Submit Article
  • About Us
TecAdmin
Menu
  • Home
  • Ubuntu 18.04
    • Whats New?
    • Upgrade Ubuntu
    • Install Java
    • Install Node.js
    • Install Docker
    • Install Git
    • Install LAMP Stack
  • Tutorials
    • AWS
    • Shell Scripting
    • Docker
    • Git
    • MongoDB
  • Funny Tools
  • FeedBack
  • Submit Article
  • About Us

How to Mount S3 Bucket on CentOS/RHEL and Ubuntu using S3FS

Written by Rahul, Updated on May 29, 2019

S3FS is FUSE (File System in User Space) based solution to mount an Amazon S3 buckets, We can use system commands with this drive just like as another Hard Disk in the system. On s3fs mounted files systems we can simply use cp, mv and ls the basic Unix commands similar to run on locally attached disks.

If you like to access S3 buckets without mounting on system, use s3cmd command line utility to manage s3 buckets. s3cmd is also provides faster speed for data upload and download rather than s3fs. To work with s3cmd use next articles to install s3cmd in Linux systems and Windows systems.

amazon-banner

This article will help you to install S3FS and Fuse by compiling from source, and also help you to mount S3 bucket on your CentOS/RHEL and Ubuntu systems.

Step 1 – Remove Existing Packages

First, check if you have any existing s3fs or fuse package installed on your system. If installed it already remove it to avoid any file conflicts.

### CentOS and RedHat Systems ###
yum remove fuse fuse-s3fs

### Ubuntu Systems ### 
sudo apt-get remove fuse

Step 2: Install Required Packages

After removing packages. First, we will install all the dependencies for fuse and s3cmd. Install the required packages to system use following command.

### CentOS and RedHat Systems ###
yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap

### Ubuntu Systems ### 
sudo apt-get install build-essential libcurl4-openssl-dev libxml2-dev mime-support

Step 3 – Download and Compile Fuse

Download and compile latest version of fuse source code. For this article, we are using fuse version 3.5. Following the set of command will compile fuse and add fuse module in the kernel.

cd /usr/src/
wget https://github.com/libfuse/libfuse/releases/download/fuse-3.5.0/fuse-3.5.0.tar.xz
tar Jxf fuse-3.5.0.tar.xz
cd fuse-3.1.0
./configure --prefix=/usr/local
make && make install
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
ldconfig
modprobe fuse

Step 4 – Download and Compile Latest S3FS

Download and compile latest version of s3fs source code. For this article we are using s3fs version 1.74. After downloading extract the archive and compile source code in system.

cd /usr/src/
git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure
make && make install

Step 5 – Setup Access Key

Also In order to configure s3fs, we would require Access Key and Secret Key of your S3 Amazon account. Get these security keys from Here.

echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > ~/.passwd-s3fs
chmod 600 ~/.passwd-s3fs

Note: Change AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with your actual key values.

Step 6 – Mount S3 Bucket

Finally mount your s3 bucket using following set of commands. For this example, we are using s3 bucket name as mydbbackup and mount point as /s3mnt.

mkdir /tmp/cache /s3mnt
chmod 777 /tmp/cache /s3mnt
s3fs -o use_cache=/tmp/cache mydbbackup /s3mnt

Share it!
Share on Facebook
Share on Twitter
Share on LinkedIn
Share on Reddit
Share on Tumblr
Share on Whatsapp
Rahul
Rahul
Connect on Facebook Connect on Twitter

I, Rahul Kumar am the founder and chief editor of TecAdmin.net. I am a Red Hat Certified Engineer (RHCE) and working as an IT professional since 2009..

38 Comments

  1. Avatar donnyO Reply
    August 19, 2020 at 6:01 pm

    any idea how to mount local S3 (not on AWS) created on PURE storage. I got the bucket name, Bucket Endpoint, Bucket Access Key and bucket Secret Access Key .

  2. Avatar Herb Gilliland Reply
    December 9, 2019 at 3:40 pm

    They use meson/ninja now, not make && make install nor ./configure

  3. Avatar Deepak Kumar Reply
    April 14, 2019 at 8:47 am

    I’m got error :

    s3fs: HTTP: 403 Forbidden – it is likely that your credentials are invalid

    are you have solved ??

  4. Avatar jj Reply
    March 3, 2019 at 2:58 am

    Can be compatible with the cheap wasabi.com instead of amazon s3?

    Thank you.

  5. Avatar Lau Seng Khung Reply
    January 19, 2018 at 3:55 am

    Steps 4

    ./configure <== this will fail because there is no configure file

    • Avatar Ronald Reply
      October 16, 2019 at 11:47 am

      Do you have a solution for this problem? Run into the same problem.
      regards Ronald

    • Avatar Yousef Reply
      October 28, 2019 at 2:13 pm

      Same here!

  6. Avatar Arkadiusz Reply
    December 28, 2016 at 7:59 am

    On Ubuntu steps 1-4 can be replaced with one command:
    sudo apt-get install s3fs
    Do this instead of steps 1-4 and continue from step 5. It works – tested.

  7. Avatar Thijs Reply
    October 25, 2016 at 10:33 am

    Do NOT use fuse3* but stick to a fuse 2*, otherwise you will not be able to install s3fs.

  8. Avatar Arunkumar Reply
    September 14, 2016 at 7:09 am

    I am getting below error.

    fuse: warning: library too old, some operations may not not work

    # rpm -qa | grep fuse
    libconfuse-2.7-4.el6.x86_64
    fuse-libs-2.8.3-5.el6.x86_64
    #

    though latest package is not available in Yum repo. please suggest

  9. Avatar Khan Reply
    June 18, 2016 at 10:09 pm

    Hello, i followed your guide (on Ubuntu 14.04, Bitnami – EC2 ) , i am getting following error at “make”
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libcurl.so: undefined reference to `[email protected]_2.4_2′
    collect2: error: ld returned 1 exit status
    make[2]: *** [s3fs] Error 1
    make[2]: Leaving directory `/tmp/s3fs-fuse-1.80/src’
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/tmp/s3fs-fuse-1.80′
    make: *** [all] Error 2

  10. Avatar Abu Kamal Reply
    April 19, 2016 at 5:04 am

    use this if you get “bash: s3fs: command not found”
    sudo /usr/local/bin/s3fs -o use_cache=/tmp/cache mydbbackup /s3mnt

  11. Avatar Tom Fazakas Reply
    April 14, 2016 at 1:05 pm

    Those of you having an error on CentOS: s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

    yum install fuse-libs

  12. Avatar James Reply
    February 27, 2016 at 12:22 pm

    Great article! Worked a treat….

    only issue was my library path required updating as when I ran s3fs it couldn’t find the fuse dependancies.

    LD_LIBRARY_PATH=/usr/local/lib
    export LD_LIBRARY_PATH

    That did the trick 🙂

  13. Avatar Bhavesh Reply
    May 6, 2015 at 8:17 am

    Hi..
    i have mounted s3 sucessfully , but when i try to “cd ” to mounted dirs it says “operation not permitted”

    cd: app_logs/: Operation not permitted

    below is permission for dir

    d———. 1 root root 1 May 6 2015 app_logs

    below command used
    s3fs -o use_cache=/tmp/cache s3bukcket /s3mnt

    what permission i need to set for bucket or bucket folder ?
    am i writing wrong command for mount?

  14. Avatar Mehul Reply
    March 31, 2015 at 11:16 am

    Hi Rahul,

    Can you please share the steps to mount Amazon S3 bucket on windows OS for both 2008 & 2012?

    Thanks & Regards,
    Mehul

  15. Avatar Mukta Reply
    December 26, 2014 at 12:08 pm

    Thanks! This was quite helpful.
    You can take a look at the docker image which I built, with S3FS and S3 bucket mounting capabilities:
    https://registry.hub.docker.com/u/ihealthtechnologies/s3-mount/

  16. Avatar David Reply
    December 23, 2014 at 2:37 pm

    Hi all, I’m receiving the following error after trying to mount the bucker:

    # s3fs -o user_cache=/tmp/cache lehar-backup /s3mnt
    s3fs: /lib/libfuse.so.2: version `FUSE_2.8′ not found (required by s3fs)

    I downloaded fuse 2.8 and compiled it per the instructions however if there is something I’m missing I’d love to know what that is. Let me know what information you might need. Thanks for any help you can provide.

    ~Dave

  17. Avatar Srikanth Reply
    December 23, 2014 at 11:07 am

    My disk is full after mount and not able to use the system. Ideally it should not take disk space.
    How can i resolve the issue?

    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/xvda1 10321208 10297608 0 100% /
    none 847476 0 847476 0% /dev/shm
    s3fs 274877906944 0 274877906944 0% /s3mnt

  18. Avatar Raju Reply
    December 17, 2014 at 11:38 am

    I’m got error :

    s3fs: HTTP: 403 Forbidden – it is likely that your credentials are invalid

    are you have solved ??

  19. Avatar manoj Reply
    December 11, 2014 at 4:08 pm

    I am getting error:
    s3fs: unable to access MOUNTPOINT storingfiles1: No such file or directory

  20. Avatar Murthi Reply
    November 28, 2014 at 7:08 am

    Hi Team,
    Very good it working fine in my ubuntu system

  21. Avatar Anonimo Reply
    October 28, 2014 at 4:52 pm

    So far so good but I get this error:

    [[email protected] s3fs-1.74]# s3fs -o use_cache=/tmp/cache agarta /etc/httpd/imagestore/
    bash: s3fs: command not found

    any idea on how to resolve this?

  22. Avatar Vineshkumar Reply
    August 22, 2014 at 5:24 am

    Hi Rahul,

    Thanks for the tutorial. All worked fine, except this messsage – Transport endpoint is not connected. Unmounted and mounted again using the link you shared in comments. I am using s3fs to connect to Google Cloud Storage.

    Kindly share some pointers or might be issue with the permission level mentioned above.

  23. Avatar Bruno Reply
    August 13, 2014 at 1:32 am

    Willian

    do it:

    # vim /etc/ld.so.conf

    /usr/local/lib
    include ld.so.conf.d/*.conf

    # ldconfig
    # s3fs -o use_cache=/tmp/cache ****.******.*** /s3mnt

  24. Avatar Bruno Reply
    August 12, 2014 at 11:26 pm

    s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

  25. Avatar William Reply
    August 1, 2014 at 8:57 pm

    I have a problem, can you helpme?

    [[email protected]*********** s3fs-1.74]# s3fs -o use_cache=/tmp/cache ****.******.*** /s3mnt
    s3fs: error while loading shared libraries: libfuse.so.2: cannot open shared object file: No such file or directory

    • Avatar kautilya Reply
      August 21, 2014 at 8:05 am

      hi

      install fuse-libs and it should solve your issue… it did in my case…

      sudo yum install fuse-libs

      hope that helps

      thanks

      • Avatar Damien Reply
        August 5, 2015 at 1:40 pm

        yes, you could add this in the tutorial ! it solved it in my case too. Thanks

      • Avatar ila Reply
        July 19, 2016 at 11:03 am

        thanks!

  26. Avatar bo Reply
    June 23, 2014 at 11:20 pm

    Followed this tutorial – doesn’t work:

    [email protected]:/home/ubuntu# ls /s3mnt
    ls: cannot access /s3mnt: Transport endpoint is not connected
    [email protected]:/home/ubuntu#

    • Rahul Rahul Reply
      June 24, 2014 at 3:10 am

      Hi,

      Try to umount and mount s3bucket again and wait for few time to sync. Also visit this link.

      • Avatar bo Reply
        June 24, 2014 at 5:05 pm

        The problem was permissions on S3 bucket. Maybe you could add some recommendations on this subject? It would make your article more complete.

        Thank you

        • Rahul Rahul Reply
          June 26, 2014 at 3:12 am

          Hi bo,

          We would be thankful for your contributions and make the article more useful for users.

          We will also check for the permission problems.

          • Avatar bo Reply
            July 3, 2014 at 9:37 pm

            I would also add info on automatic mounting the s3 bucket. Something like editing /etc/fdtab to add
            s3fs#bucketname mountpath fuse allow_other,_netdev,nosuid,nodev,url=https://s3.amazonaws.com 0 0

          • Avatar Anil A Reply
            January 5, 2015 at 1:27 am

            I have followed the tutorial and get to the state of “Transport endpoint is not connected”. Any updates on what permissions need to be set on S3 bucket?

            • Avatar Chad
              October 17, 2017 at 3:14 pm

              I had to add -o allow_other to the command line to get it to work fully.

  27. Avatar Rogerio Reply
    May 27, 2014 at 10:31 pm

    and about fstab?
    How can I put cache dir

Leave a Reply Cancel reply

Popular Posts

  • How To Install Python 3.9 on Ubuntu 20.04 5
  • How To Install Python 3.9 on Ubuntu 18.04 0
  • How to Use AppImage on Linux (Beginner Guide) 2
  • How to Install Python 3.9 on CentOS/RHEL 7 & Fedora 32/31 0
  • How To Install VNC Server on Ubuntu 20.04 1
© 2013-2020 Tecadmin.net. All Rights Reserved | Terms  | Privacy Policy