blocks|key|226414|text|There's+a+good+article+on+s3fs+here，在读完之后，我求助于EBS分享。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|226415|它强调了使用s3fs时的几个重要注意事项，即与S3的固有限制相关的事项：|226416|226417|没有文件可以超过5+GB|unordered-list-item|226418|您不能部分更新文件，因此更改单个字节将在许多小文件上重新上传整个file.|226419|operation非常有效(毕竟每个小文件都是一个单独的s3fs对象)但大文件是非常高效的|226420|+S3支持部分/分块下载，s3fs不利用这一点，因此如果您只想读取1+GB文件的一个字节，则必须下载整个GB。|226421|因此，这取决于您存储的是什么，s3fs是否是一个可行的选项。比方说，如果你正在存储照片，你想要写入或读取整个文件，而不是增量地更改文件，那么这很好，尽管有人可能会问，如果你这样做，那么为什么不直接使用S3的API呢？|226422|如果你谈论的是应用程序数据(比如数据库文件、日志文件)，而你想要做一些小的增量更改，那么它肯定是不能工作的，因为你不能增量地更改一个文件。(译者注:如果你想对文件进行增量更改，那么S3肯定不起作用。|226423|上面提到的文章确实讨论了一个类似的应用程序-+s3backer+-它通过在S3上实现虚拟文件系统来解决性能问题。这绕过了性能问题，但它本身也有一些问题：|226424|226425|数据损坏的高风险，由于延迟写入|226426|数据块大小过小(例如，默认大小为4K+)可能会增加大量额外成本(例如，对于50+it，4K数据块大小为130美元)|226427|太大数据块大小可能会增加大量数据传输和存储费用。|226428|内存使用率可能令人望而却步:默认情况下，它缓存1000个数据块。|226429|226430|对于默认的4K块大小，这不是问题，但大多数用户|226431|可能想要增加数据块大小。|226432|我求助于EBS，从EC2实例挂载、驱动、共享。但您应该知道，尽管EBS挂载的NFS共享是性能最高的选项，但它有一个很大的问题-单点故障；如果共享EBS卷的计算机宕机，则您将无法访问访问该共享的所有计算机。|226433|这是一个我能够接受的风险，也是我最终选择的选择。我希望这能帮到你。|226434|entityMap|0|LINK|mutability|MUTABLE|url|http://www.turnkeylinux.org/blog/exploring-s3-based-filesystems-s3fs-and-s3backer|1|https://code.google.com/p/s3backer/^0|0|Z|0|0|0|0|0|0|0|0|0|0|N|8|1|0|0|0|0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|1N|8|@]|9|@$A|1O|B|1P|1|1Q]]|C|$]]|$1|D|3|E|5|6|7|1R|8|@]|9|@]|C|$]]|$1|F|3|-4|5|6|7|1S|8|@]|9|@]|C|$]]|$1|G|3|H|5|I|7|1T|8|@]|9|@]|C|$]]|$1|J|3|K|5|I|7|1U|8|@]|9|@]|C|$]]|$1|L|3|M|5|I|7|1V|8|@]|9|@]|C|$]]|$1|N|3|O|5|6|7|1W|8|@]|9|@]|C|$]]|$1|P|3|Q|5|6|7|1X|8|@]|9|@]|C|$]]|$1|R|3|S|5|6|7|1Y|8|@]|9|@]|C|$]]|$1|T|3|U|5|6|7|1Z|8|@]|9|@$A|20|B|21|1|22]]|C|$]]|$1|V|3|-4|5|6|7|23|8|@]|9|@]|C|$]]|$1|W|3|X|5|I|7|24|8|@]|9|@]|C|$]]|$1|Y|3|Z|5|I|7|25|8|@]|9|@]|C|$]]|$1|10|3|11|5|I|7|26|8|@]|9|@]|C|$]]|$1|12|3|13|5|I|7|27|8|@]|9|@]|C|$]]|$1|14|3|-4|5|6|7|28|8|@]|9|@]|C|$]]|$1|15|3|16|5|6|7|29|8|@]|9|@]|C|$]]|$1|17|3|18|5|6|7|2A|8|@]|9|@]|C|$]]|$1|19|3|1A|5|6|7|2B|8|@]|9|@]|C|$]]|$1|1B|3|1C|5|6|7|2C|8|@]|9|@]|C|$]]|$1|1D|3|-4|5|6|7|2D|8|@]|9|@]|C|$]]]|1E|$1F|$5|1G|1H|1I|C|$1J|1K]]|1L|$5|1G|1H|1I|C|$1J|1M]]]]

<a href="http://www.turnkeylinux.org/blog/exploring-s3-based-filesystems-s3fs-and-s3backer">There's a good article on s3fs here</a>, which after reading I resorted to an EBS Share.

It highlights a few important considerations when using s3fs, namely related to the inherent limitations of S3:

<ul>
<li>no file can be over 5GB </li>
<li>you can't partially update a file so changing a single byte will re-upload the entire file.</li>
<li>operation on many small files are very efficient (each is a separate S3 object after all) but large files are very inefficient</li>
<li>Though S3 supports partial/chunked downloads, s3fs doesn't take advantage of this so if you want to read just one byte of a 1GB file, you'll have to download the entire GB.</li>
</ul>

It therefore depends on what you are storing whether s3fs is a feasible option. If you're storing say, photos, where you want to write an entire file or read an entire file never incrementally change a file, then its fine, although one may ask, if you're doing this, then why not just use S3's API Directly?

If you're talking about appliation data, (say database files, logging files) where you want to make small incremental change then its a definite no - S3 Just doesn't work that way you can't incrementally change a file.

The article mentioned above does talk about a similar application - <a href="https://code.google.com/p/s3backer/">s3backer</a> - which gets around the performance issues by implementing a virtual filesystem over S3. This gets around the performance issues but itself has a few issues of its own:

<ul>
<li>High risk for data corruption, due to the delayed writes </li>
<li>too small block sizes (e.g., the 4K default) can add significant
extra costs (e.g., $130 for 50GB with 4K blocks worth of storage)</li>
<li>too large block sizes can add significant data transfer and storage 
fees.</li>
<li>memory usage can be prohibitive: by default it caches 1000 blocks. 
With the default 4K block size that's not an issue but most users 
will probably want to increase block size.</li>
</ul>

I resorted to EBS Mounted Drived shared from an EC2 instance. But you should know that although the most performant option it has one big problem
An EBS Mounted NFS Share has its own problems - a single point of failure; if the machine that's sharing the EBS Volume goes down then you lose access on all machines which access the share.

This is a risk I was able to live with and was the option I chose in the end. I hope this helps.

blocks|key|226450|text|这是一个老问题，所以我将与S3FS分享我过去一年的经验。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|226451|最初，它有许多bug和内存泄漏(我有一个cron-job来每2个小时重新启动它)，但在最新的1.73版本中，它非常稳定。|226452|S3FS最大的好处是你少了一件事要操心，还能免费获得一些性能上的好处。|226453|您的大多数S3请求将是PUT+(~5%25)和GET+(~95%25)。如果您不需要任何后处理(例如，缩略图生成)。如果你不需要任何后处理，你不应该首先访问你的web服务器并直接上传到S3+(使用CORS)。|226454|假设你正在访问服务器，这可能意味着你需要对图像进行一些后处理。使用S3应用程序接口，您将上传到服务器，然后再上传到S3。如果用户想裁剪，你需要再次从S3下载，然后重新上传到服务器，裁剪，然后上传到S3。在启用S3FS和本地缓存的情况下，将为您处理此编排并保存从S3下载的文件。|226455|在缓存方面，如果您将缓存到EC2上的临时驱动器，您将获得与out一起带来的性能优势，并且可以清除缓存，而无需担心任何事情。除非磁盘空间用完，否则应该没有理由清除缓存。这使得像搜索和过滤这样的遍历操作变得更加容易。|226456|我希望它能与S3+(RSync风格)完全同步。这将使其成为企业版的DropBox或用于S3的Google+Drive，但不必为配额和随之而来的费用而战。|226457|entityMap^0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|P|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Q|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|R|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|S|8|@]|9|@]|A|$]]|$1|H|3|I|5|6|7|T|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|U|8|@]|9|@]|A|$]]|$1|L|3|M|5|6|7|V|8|@]|9|@]|A|$]]|$1|N|3|-4|5|6|7|W|8|@]|9|@]|A|$]]]|O|$]]

This is an old question so I'll share my experience over the past year with S3FS.

Initially, it had a number of bugs and memory leaks (I had a cron-job to restart it every 2 hours) but with the latest release 1.73 it's been very stable.

The best thing about S3FS is you have one less things to worry about and get some performance benefits for free.

Most of your S3 requests are going to be PUT (~5%) and GET (~95%). If you don't need any post-processing (thumbnail generation for example). If you don't need any post-processing, you shouldn't be hitting your web server in the first place and uploading directly to S3 (using CORS).

Assuming you are hitting the server probably means you need to do some post-processing on images. With an S3 API you'll be uploading to the server, then uploading to S3. If the user wants to crop, you'll need to download again from S3, then re-upload to server, crop and then upload to S3. With S3FS and local caching turned on this orchestration is taken care of for you and saves downloading files from S3.

On caching, if you are caching to an ephemeral drive on EC2, you get a the performance benefits that come with out and can purge your cache without having to worry about anything. Unless you run out of disk space, you should have no reason to purge your cache. This makes traversing operations like searching and filtering much easier.

The one thing I do wish it has was full sync with S3 (RSync style). That would make it an enterprise version of DropBox or Google Drive for S3 but without having to contend with the quotas and fees that come with it.

How stable is s3fs to mount an Amazon S3 bucket as a local directory in linux? Is it recommended/stable for high demand production environments?

Are there any better/similar solutions?

Update: Would it be better to use EBS and to mount it via NFS to all other AMIs?

How stable is s3fs to mount an Amazon S3 bucket as a local directory

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

EdgeOne AI 安全实战专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

在s3fs中，将亚马逊S3存储桶挂载为本地目录的稳定性如何？对于高需求的生产环境，它是否推荐/稳定？有没有更好的/类似的解决方案？更新：使用EBS并通过NFS将其挂载到所有其他AMI是否更好？

问s3fs将亚马逊S3存储桶挂载为本地目录的稳定性如何
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问s3fs将亚马逊S3存储桶挂载为本地目录的稳定性如何EN