blocks|key|3365392|text|除了推荐的SAX解析之外，您还可以使用StAX+(包javax.xml.stream+)中包含的JDK+(某种SAX演变)。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3365393|http://stax.codehaus.org/Home|offset|length|3365394|Brief简介：http://www.xml.com/pub/a/2003/09/17/stax.html|unordered-list-item|3365395|
|3365396|Javadoc：https://docs.oracle.com/javase/8/docs/api/javax/xml/stream/package-summary.html|3365397|
+|3365398|StAX项目主页|3365399|3365400|entityMap|0|LINK|mutability|MUTABLE|url|1|http://www.xml.com/pub/a/2003/09/17/stax.html|2|https://docs.oracle.com/javase/8/docs/api/javax/xml/stream/package-summary.html^0|0|0|T|0|0|8|19|1|0|0|8|27|2|0|0|0|0^^$0|@$1|2|3|4|5|6|7|12|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|13|8|@]|9|@$D|14|E|15|1|16]]|A|$]]|$1|F|3|G|5|H|7|17|8|@]|9|@$D|18|E|19|1|1A]]|A|$]]|$1|I|3|J|5|6|7|1B|8|@]|9|@]|A|$]]|$1|K|3|L|5|H|7|1C|8|@]|9|@$D|1D|E|1E|1|1F]]|A|$]]|$1|M|3|N|5|6|7|1G|8|@]|9|@]|A|$]]|$1|O|3|P|5|H|7|1H|8|@]|9|@]|A|$]]|$1|Q|3|-4|5|6|7|1I|8|@]|9|@]|A|$]]|$1|R|3|-4|5|6|7|1J|8|@]|9|@]|A|$]]]|S|$T|$5|U|V|W|A|$X|C]]|Y|$5|U|V|W|A|$X|Z]]|10|$5|U|V|W|A|$X|11]]]]

Aside the recommended SAX parsing, you could use the StAX API (kind of a SAX evolution), included in the JDK (package javax.xml.stream ).

<ul>
<li>StAX Project Home: <a href="http://stax.codehaus.org/Home" rel="nofollow">http://stax.codehaus.org/Home</a></li>
<li>Brief introduction: <a href="http://www.xml.com/pub/a/2003/09/17/stax.html" rel="nofollow">http://www.xml.com/pub/a/2003/09/17/stax.html</a></li>
<li>Javadoc: <a href="https://docs.oracle.com/javase/8/docs/api/javax/xml/stream/package-summary.html" rel="nofollow">https://docs.oracle.com/javase/8/docs/api/javax/xml/stream/package-summary.html</a></li>
</ul>

blocks|key|4776105|text|使用基于SAX的解析器，它以事件流的形式向您显示文档的内容。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4776106|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

Use a SAX based parser that presents you with the contents of the document in a stream of events.

blocks|key|1944484|text|与SAX相比，StAX应用编程接口更容易处理。这是一个short+tutorial|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1944485|entityMap|0|LINK|mutability|MUTABLE|url|http://tutorials.jenkov.com/java-xml/stax.html^0|R|E|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

StAX API is easier to deal with compared to SAX. Here is a <a href="http://tutorials.jenkov.com/java-xml/stax.html" rel="nofollow">short tutorial</a>

blocks|key|1944391|text|将文件流式传输到SAX解析器，并将其以块的形式读取到内存中。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1944392|SAX为您提供了大量的控制，并且事件驱动是有意义的。api有点难以掌握，您必须注意一些事情，比如当characters()方法被调用时，但基本思想是编写一个内容处理程序，在读取每个xml元素的开始和结束时调用该处理程序。因此，您可以跟踪文档中的当前xpath，确定哪些路径包含您感兴趣的数据，并确定哪个路径标记您要保存、传递或以其他方式处理的块的末尾。|1944393|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

Stream the file into a SAX parser and read it into memory in chunks.

SAX gives you a lot of control and being event-driven makes sense. The api is a little hard to get a grip on, you have to pay attention to some things like when the characters() method is called, but the basic idea is you write a content handler that gets called when the start and end of each xml element is read. So you can keep track of the current xpath in the document, identify which paths have which data you're interested in, and identify which path marks the end of a chunk that you want to save or hand off or otherwise process.

blocks|key|1949082|text|试试VTD-XML。我发现它比SAX性能更好，更重要的是，更容易使用。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1949083|entityMap|0|LINK|mutability|MUTABLE|url|http://vtd-xml.sourceforge.net/^0|2|7|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

Try <a href="http://vtd-xml.sourceforge.net" rel="nofollow">VTD-XML</a>. I've found it to be more performant, and more importantly, easier to use than SAX.

blocks|key|4776185|text|正如其他人所说，使用SAX解析器，因为它是一个流解析器。使用各种事件，您可以根据需要提取信息，然后动态地将其存储在其他位置(数据库、另一个文件等)。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4776186|如果您确实只需要一个较小的子集，或者只是对文件进行摘要，您甚至可以将其存储在内存中。当然，这取决于用例。|4776187|如果您要假脱机到数据库，请务必小心使您的进程可重启或其他。在1.8+in中可能会发生很多事情，但在中间可能会失败。|4776188|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|H|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|I|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|J|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|G|$]]

As others have said, use a SAX parser, as it is a streaming parser. Using the various events, you extract your information as necessary and then, on the fly store it someplace else (database, another file, what have you).

You can even store it in memory if you truly just need a minor subset, or if you're simply summarizing the file. Depends on the use case of course.

If you're spooling to a DB, make sure you take some care to make your process restartable or whatever. A lot can happen in 1.8GB that can fail in the middle.

blocks|key|1944368|text|使用几乎所有的SAX+Parser一次只传输一点文件。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1944369|entityMap|0|LINK|mutability|MUTABLE|url|http://www.saxproject.org/|1|http://download.oracle.com/javase/1.4.2/docs/api/org/xml/sax/package-summary.html^0|7|3|0|B|6|1|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@$A|O|B|P|1|Q]|$A|R|B|S|1|T]]|C|$]]|$1|D|3|-4|5|6|7|U|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]|L|$5|G|H|I|C|$J|M]]]]

Use almost any <a href="http://www.saxproject.org/" rel="nofollow">SAX</a> <a href="http://download.oracle.com/javase/1.4.2/docs/api/org/xml/sax/package-summary.html" rel="nofollow">Parser</a> to stream the file a bit at a time.

blocks|key|3365512|text|我遇到了一个类似的问题--我必须读取整个XML文件并在内存中创建一个数据结构。在这个数据结构上(整个东西都必须加载)，我必须做各种操作。很多XML元素都包含文本(我必须在输出文件中输出文本，但对于算法来说并不重要)。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3365513|FIrstly，正如这里所建议的，我使用SAX解析文件并构建数据结构。我的文件是4+6GB的，我有一台8+6GB的机器，所以我想可能有3+6GB的文件只是文本，而java.lang.String可能需要6+6GB的UTF-16来存储这些文本。|3365514|如果JVM占用的空间比计算机的物理RAM多，那么计算机将进行交换。执行mark%2Bsweep垃圾收集将导致以随机顺序访问页面，并将对象从一个对象池移动到另一个对象池，这基本上会杀死机器。|3365515|因此，我决定将所有字符串写到磁盘的一个文件中(+FS显然可以很好地处理3+3GB的顺序写入，并且当在操作系统中读取它时，将使用可用内存作为文件系统缓存；可能仍然会有随机访问读取，但比java中的GC要少)。我创建了一个小助手类，如果它对您有帮助，欢迎您下载：StringsFile+javadoc+%7C+Download+ZIP。|offset|length|3365516|StringsFile+file+=+new+StringsFile();
StringInFile+str+=+file.newString("abc");++++++++//+writes+string+to+file
System.out.println("str+is:+"+%2B+str.toString());+//+fetches+string+from+file|code-block|syntax|javascript|3365517|entityMap|0|LINK|mutability|MUTABLE|url|http://www.databasesandlife.com/blog-attachments/databasesandlife-util-javadoc/com/databasesandlife/util/StringsFile.html|1|http://www.databasesandlife.com/blog-attachments/databasesandlife-java-util.zip^0|0|0|0|3L|J|0|47|C|1|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Z|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|10|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|11|8|@]|9|@$H|12|I|13|1|14]|$H|15|I|16|1|17]]|A|$]]|$1|J|3|K|5|L|7|18|8|@]|9|@]|A|$M|N]]|$1|O|3|-4|5|6|7|19|8|@]|9|@]|A|$]]]|P|$Q|$5|R|S|T|A|$U|V]]|W|$5|R|S|T|A|$U|X]]]]

I had a similar problem - I had to read a whole XML file and create a data structure in memory. On this data structure (the whole thing had to be loaded) I had to do various operations. A lot of the XML elements contained text (which I had to output in my output file, but wasn't important for the algorithm).

FIrstly, as suggested here, I used SAX to parse the file and build up my data structure. My file was 4GB and I had an 8GB machine so I figured maybe 3GB of the file was just text, and java.lang.String would probably need 6GB for those text using its UTF-16.

If the JVM takes up more space than the computer has physical RAM, then the machine will swap. Doing a mark+sweep garbage collection will result in the pages getting accessed in a random-order manner and also objects getting moved from one object pool to another, which basically kills the machine.

So I decided to write all my strings out to disk in a file (the FS can obviously handle sequential-write of the 3GB just fine, and when reading it in the OS will use available memory for a file-system cache; there might still be random-access reads but fewer than a GC in java). I created a little helper class which you are more than welcome to download if it helps you: <a href="http://www.databasesandlife.com/blog-attachments/databasesandlife-util-javadoc/com/databasesandlife/util/StringsFile.html" rel="nofollow">StringsFile javadoc</a> | <a href="http://www.databasesandlife.com/blog-attachments/databasesandlife-java-util.zip" rel="nofollow">Download ZIP</a>.

<pre><code>StringsFile file = new StringsFile();
StringInFile str = file.newString("abc"); // writes string to file
System.out.println("str is: " + str.toString()); // fetches string from file
</code></pre>

blocks|key|4776365|text|StaX为%2B1。它比SaX更容易使用，因为你不需要写回调(基本上你只需要循环while的所有元素，直到你完成)，并且它对它可以处理的文件大小没有限制。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4776366|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

+1 for StaX. It's easier to use than SaX because you don't need to write callbacks (you essentially just loop over all elements of the while until you're done) and it has (AFAIK) no limit as to the size of the files it can process.

I need a xml parser to parse a file that is approximately 1.8 gb. 
So the parser should not load all the file to memory.

Any suggestions?

Java XML Parser for huge files

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我需要一个xml解析器来解析一个大约1.8 gb的文件。因此，解析器不应该将所有文件加载到内存中。有什么建议吗？

问面向大型文件的Java XML解析器
EN

回答 9

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问面向大型文件的Java XML解析器EN

回答 9

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问面向大型文件的Java XML解析器
EN