blocks|key|2228297|text|块大小64+an意味着块的上限大小。这并不意味着小于64‘t的文件块将消耗64’t。它不会消耗64+1MB来存储1MB的块。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2228298|如果文件是160兆字节的，|offset|length|style|BOLD|2228299|📷|atomic|2228300|​|2228301|希望这能有所帮助。|2228302|entityMap|0|IMAGE|mutability|IMMUTABLE|imageUrl|https://developer.qcloudimg.com/http-save/yehe-900000/029431ea83fb7137e23857d86493b06a.png|imageAlt^0|0|5|7|0|0|1|0|0|0|0^^$0|@$1|2|3|4|5|6|7|X|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Y|8|@$D|Z|E|10|F|G]]|9|@]|A|$]]|$1|H|3|I|5|J|7|11|8|@]|9|@$D|12|E|13|1|14]]|A|$]]|$1|K|3|L|5|6|7|15|8|@]|9|@]|A|$]]|$1|M|3|N|5|6|7|16|8|@]|9|@]|A|$]]|$1|O|3|-4|5|6|7|17|8|@]|9|@]|A|$]]]|P|$Q|$5|R|S|T|A|$U|V|W|-4]]]]

Block size 64MB means an upper bound size for a block. It doesn't mean that file blocks less than 64MB will consume 64MB. It will not consume 64MB to store a chunk of 1MB.

If the file is 160 megabytes,
<a href="https://i.stack.imgur.com/pALnI.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pALnI.png" alt="enter image description here"></a>

Hope this helps.

blocks|key|2228339|text|根据此页的说法。看起来是一个64+MB的块和一个1MB的块。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|2228340|HDFS常常幸灾乐祸地不知道，一个块中的最后记录可能只是部分记录，其其余内容被分流到下面的块。HDFS只想确保文件被分割成大小相等的块，这些块与Hadoop实例的预定义块大小相匹配.并不是您需要存储的每个文件都是系统块大小的确切倍数，因此文件的最终数据块只占用所需的空间。|blockquote|2228341|entityMap|0|LINK|mutability|MUTABLE|url|http://m.dummies.com/how-to/content/data-blocks-in-the-hadoop-distributed-file-system-.html^0|2|2|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@]|9|@$A|P|B|Q|1|R]]|C|$]]|$1|D|3|E|5|F|7|S|8|@]|9|@]|C|$]]|$1|G|3|-4|5|6|7|T|8|@]|9|@]|C|$]]]|H|$I|$5|J|K|L|C|$M|N]]]]

According to <a href="http://m.dummies.com/how-to/content/data-blocks-in-the-hadoop-distributed-file-system-.html" rel="nofollow">this page</a>. Looks like it'll be one 64 MB block and one 1 MB block. 

<blockquote>
 HDFS is often blissfully unaware that the final record in one block may be only a partial record, with the rest of its content shunted off to the following block. HDFS only wants to make sure that files are split into evenly sized blocks that match the predefined block size for the Hadoop instance... Not every file you need to store is an exact multiple of your system’s block size, so the final data block for a file uses only as much space as is needed.
</blockquote>

blocks|key|2784809|text|答案是两个块，一个64+1MB和其他1MB。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2784810|与其他文件系统一样，HDFS将文件拆分为块，然后将这些块保存到磁盘中。|2784811|但它们之间有两个主要区别：|2784812|HDFS块大小很大，因为每个块在namenode上都有一个元数据记录，较小的块大小意味着大量块和namenode的元数据重载。|ordered-list-item|2784813|因此，在HDFS中使用更大的块大小。|2784814|HDFS块大小只是linux文件系统上的一个抽象，因此65+1MB将使用一个64+1MB块和来自第二个块的其他1MB空间，来自第二个块的rest+63MB仍然是免费的，可以用于其他数据。|2784815|也就是说，Namenode将为65+is记录两个块，但实际的文件系统空间仅为65+is。|2784816|entityMap^0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|R|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|S|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|T|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|U|8|@]|9|@]|A|$]]|$1|K|3|L|5|H|7|V|8|@]|9|@]|A|$]]|$1|M|3|N|5|6|7|W|8|@]|9|@]|A|$]]|$1|O|3|-4|5|6|7|X|8|@]|9|@]|A|$]]]|P|$]]

The answer is 2 blocks, one 64MB and other 1MB.

HDFS just like other filesystems splits the file into blocks and then saves those blocks to disks.

But there are two major differences between them :

<ol>
<li>HDFS block sizes are huge because every block has a metadata record at namenode, smaller block sizes means a lot of blocks and overloading of namenode with metadata. </li>
</ol>

Hence, bigger block sizes used in HDFS.

<ol start="2">
<li>HDFS block sizes are just an abstraction on the linux based file system, hence 65MB will use one 64MB block and other 1MB space from second block, rest 63MB from second block is still free and available for other data.</li>
</ol>

That is, Namenode will have two blocks recorded for 65MB but the actual file system space is 65MB only.

My file size is 65MB and default hdfs block size(64MB), then how many 64MB blocks will be allotted to my file?

Is it like 1-64MB block, 1-1MB block or 2-64MB blocks? If it is 2-64MB blocks is it going to be wasted rest of the 63MB or will it be allocated to other file?

How does HDFS manage block size?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我的文件大小是65 My，默认的hdfs块大小(64 my )，那么将有多少64 my块分配给我的文件？它是像1-64MB块，1-1MB块还是2-64MB块？如果它是2-64MB块，那么它将被浪费掉其余的63 of，还是会分配给其他文件？

问HDFS如何管理块大小？
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问HDFS如何管理块大小？EN