blocks|key|2850101|text|如果您确实有大量的文件，您可能会考虑派生一个外部进程来执行搜索。如果您在类unix服务器(如linux)上运行，则可以通过使用"find“命令为您执行搜索并解析其输出来获得速度提升。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2850102|你可以在谷歌上搜索到许多如何使用"find“的例子。|2850103|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

If you really have loads of files, you might think about spawning an external process to do the searching. If you're running on a unix-like server (like linux), you might get speed gains by having the "find" command do the searching for you, and parse its output. 

You can google for many examples of how to use "find".

blocks|key|2850131|text|我认为你的servlet运行缓慢是因为硬盘速度太快。如果文件列表是永久性的，则应将其加载到内存中|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2850132|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

i think your servlet works slow because of hard drive speed. if list of files a permanent you should load it to memory

blocks|key|1441298|text|磁盘访问很慢，并且随着文件和目录数量的增加，当使用传统的递归搜索目录树时，您的servlet将很快变得无用。如果有大量并发用户同时执行相同的搜索，您将特别快地达到此限制。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1441299|相反，使用外部批处理作业来生成文件列表要好得多，然后可以通过数据库调用将这些文件读取到servlet中，甚至只需解析包含换行符分隔的所有文件名的文件即可。在Linux上使用"find“是一种简单的方法:例如|1441300|find+<path_to_directory>+-name+'*.bin'+>+list_files.txt|code-block|syntax|javascript|1441301|这将列出特定目录中以.bin结尾的每个文件名，并将其输出到名为list_files.txt的文件中。然后，您的servlet可以读取该文件并从中创建文件列表。|1441302|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|N|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|O|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|P|8|@]|9|@]|A|$]]|$1|K|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|L|$]]

Disk access is slow and as the number of files and directories increases, you'll rapidly reach a point where your servlet will be useless when using the conventional recursive search through the directory tree. You'll especially reach this limit quickly if you have a large number of concurrent users performing the same search at the same time.

It's instead, much better to use an external batch job to generate the list of files which can then be read into the servlet through a database call or even by just parsing a file containing all the file names separated by a newline character. Using "find" on Linux is a simple way to do this: e.g. 

<pre><code>find &lt;path_to_directory&gt; -name '*.bin' &gt; list_files.txt
</code></pre>

This would list every file name that ends with .bin in a particular directory and output it into a file named list_files.txt. Your servlet could then read in that file and create the list of files from there.

blocks|key|2850213|text|我认为这一过程可能进展缓慢的原因有两个：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2850214|1)磁盘I/O时间太长。这将是一个你无能为力的真正的约束。通常，操作系统非常擅长在内存中保留结构，使其能够更快地找到文件夹中的文件。如果它太慢了，你可能不得不自己在内存中建立一个索引。这完全取决于你是怎么做的。|2850215|在任何情况下，如果这就是问题所在(你可以尝试测量)，那么做过滤客户端是不会有帮助的，因为无论你在哪里做，这应该不会花很长时间。相反，您将通过向客户端发送更多数据来排序，从而使客户端变得更慢。|2850216|2)你的目录遍历有问题。你说它是“递归的”。如果你的意思是它实际上是递归的，即每当它遇到一个新的目录就会调用它自己的方法，那么这很可能会减慢你的速度(开销真的会增加)。在维基百科上有一些关于tree+traversal的东西，但基本上只是使用队列或堆栈来跟踪您在遍历中所处的位置，而不是使用您的方法状态来执行此操作。|offset|length|2850217|请注意，文件系统实际上不是树，但我假设它在本例中是树。不然的话，它会变得更多毛。|2850218|我不同意其他帖子所说的，你不能在过程中实现它。在某种程度上，它应该工作得很好，现在还不需要批处理作业。|2850219|entityMap|0|LINK|mutability|MUTABLE|url|http://en.wikipedia.org/wiki/Tree_traversal^0|0|0|0|2N|E|0|0|0|0^^$0|@$1|2|3|4|5|6|7|V|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|W|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|X|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|Y|8|@]|9|@$H|Z|I|10|1|11]]|A|$]]|$1|J|3|K|5|6|7|12|8|@]|9|@]|A|$]]|$1|L|3|M|5|6|7|13|8|@]|9|@]|A|$]]|$1|N|3|-4|5|6|7|14|8|@]|9|@]|A|$]]]|O|$P|$5|Q|R|S|A|$T|U]]]]

I see two possible reasons why this process might be going slowly:

1) Disk I/O is taking too long. This'll be a real constraint that you can't do much about. Usually the operating system is pretty good at keeping structures in memory that allow it to find files in your folders much quicker. If it is too slow regardless, you might have to build an index yourself in memory. This all depends on how you're doing it. 

In any case, if this is the issue (you can try measuring), then there's no way doing the filtering client side will help, as that shouldn't really take very long, no matter where you do it. Instead you're going to make the client slower by sending it more data to sort through.

2) There's something wrong with your directory traversal. You say it's "recursive". If you mean it's actually recursive, i.e. a method that calls itself whenever it encounters a new directory, then that might well be slowing you down (the overhead really adds up). There's some stuff about <a href="http://en.wikipedia.org/wiki/Tree_traversal" rel="nofollow noreferrer">tree traversal</a> on wikipedia, but basically just use a queue or stack to keep track of where you are in the traversal, instead of using your method state to do so.

Note that a file system isn't actually a tree, but I'm assuming that it is in this case. It gets a bit hairier otherwise.

I don't agree with the other posters that you can't implement it in-process. It should work pretty well up to a certain point, no need for batch jobs just yet.

I am writing a servlet which will examine a directory on the server (external to the web container), and recursively search for certain files (by certain files, I mean files that are of a certain extension as well as a certain naming convention). Once these files are found, the servlet responds with a long list of all of the found files (including the full path to the files). My problem is that there are so many files and directories that my servlet goes extremely slow. I was wondering if there was a best practice or existing servlet for this type of problem? Would it be more efficient to simply compile the entire list of files and do the filtering via js/jquery on the client side?

What is the most efficient way to list all of the files in a directory (including sub-directories)?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我正在编写一个servlet，它将检查服务器上的一个目录(在web容器之外)，并递归地搜索某些文件(通过某些文件，我指的是具有特定扩展名和特定命名约定的文件)。一旦找到这些文件，servlet就会返回一个包含所有找到的文件的长列表(包括文件的完整路径)。我的问题是有太多的文件和目录，以至于我的servlet运行得非常慢...

问列出目录(包括子目录)中所有文件的最有效方法是什么？
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问列出目录(包括子目录)中所有文件的最有效方法是什么？EN