我正在用C#编写一个地理信息系统客户端工具,以便从服务器检索基于GML schema (示例如下)中的“特性”。提取限制为100,000个特征。
我估计最大的extract.xml可能会达到150MB左右,所以很明显DOM解析器已经过时了,我一直在尝试在XmlSerializer和XSD.EXE生成的绑定之间做出选择--或者-- XmlReader和一个手工制作的对象图。
或者也许有一个更好的方法,我还没有考虑过?比如XLINQ,或者?
有谁能给我指点一下吗?尤其是关于任何给定方法的存储器效率。如果没有,我将不得不“原型”这两个解决方案,并并排分析它们。
我对.NET有点陌生,任何指导都将不胜感激。
谢谢你。基思。
示例-最多100,000个,每个特性最多234,600个coords。
<feature featId="27168306" fType="vegetation" fTypeId="1129" fClass="vegetation" gType="Polygon" ID="0" cLockNr="51598" metadataId="51599" mdFileId="NRM/TIS/VEGETATION/9543_22_v3" dataScale="25000">
<MultiGeometry>
<geometryMember>
<Polygon>
<outerBoundaryIs>
<LinearRing>
<coordinates>153.505004,-27.42196 153.505044,-27.422015 153.503992 .... 172 coordinates omitted to save space ... 153.505004,-27.42196</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
</geometryMember>
</MultiGeometry>
</feature>
发布于 2009-03-24 05:57:16
使用XmlReader
解析大型XML文档。XmlReader
提供了对XML数据的快速、只向前、非缓存访问。(仅向前意味着您可以从头到尾读取XML文件,但不能在文件中向后移动。) XmlReader
使用少量内存,相当于使用简单的SAX阅读器。
using (XmlReader myReader = XmlReader.Create(@"c:\data\coords.xml"))
{
while (myReader.Read())
{
// Process each node (myReader.Value) here
// ...
}
}
您可以使用XmlReader处理最大为2 GB的文件。
发布于 2009-05-14 02:39:05
Asat 14 2009年5月14日:我已经切换到使用混合方法...请参阅下面的代码。
此版本具有这两个版本的大多数优点:
* XmlReader/XmlTextReader (内存效率-->速度);以及
* XmlSerializer (代码生成-->开发的方便性和灵活性)。
它使用XmlTextReader遍历文档,并创建“doclet”,并使用XSD.EXE生成的XmlSerializer和"XML binding“类对其进行反序列化。
我猜这个食谱是普遍适用的,而且它很快...我在大约7秒内解析了一个包含56,000个GML特征的201 MB XML文档……这个应用程序的旧VB6实现需要几分钟(甚至几个小时)来解析大型摘录……所以我看起来可以走了。
再一次,a BIG感谢论坛贡献您的宝贵时间。我真的很感激。
干杯。基思。
using System;
using System.Reflection;
using System.Xml;
using System.Xml.Serialization;
using System.IO;
using System.Collections.Generic;
using nrw_rime_extract.utils;
using nrw_rime_extract.xml.generated_bindings;
namespace nrw_rime_extract.xml
{
internal interface ExtractXmlReader
{
rimeType read(string xmlFilename);
}
/// <summary>
/// RimeExtractXml provides bindings to the RIME Extract XML as defined by
/// $/Release 2.7/Documentation/Technical/SCHEMA and DTDs/nrw-rime-extract.xsd
/// </summary>
internal class ExtractXmlReader_XmlSerializerImpl : ExtractXmlReader
{
private Log log = Log.getInstance();
public rimeType read(string xmlFilename)
{
log.write(
string.Format(
"DEBUG: ExtractXmlReader_XmlSerializerImpl.read({0})",
xmlFilename));
using (Stream stream = new FileStream(xmlFilename, FileMode.Open))
{
return read(stream);
}
}
internal rimeType read(Stream xmlInputStream)
{
// create an instance of the XmlSerializer class,
// specifying the type of object to be deserialized.
XmlSerializer serializer = new XmlSerializer(typeof(rimeType));
serializer.UnknownNode += new XmlNodeEventHandler(handleUnknownNode);
serializer.UnknownAttribute +=
new XmlAttributeEventHandler(handleUnknownAttribute);
// use the Deserialize method to restore the object's state
// with data from the XML document.
return (rimeType)serializer.Deserialize(xmlInputStream);
}
protected void handleUnknownNode(object sender, XmlNodeEventArgs e)
{
log.write(
string.Format(
"XML_ERROR: Unknown Node at line {0} position {1} : {2}\t{3}",
e.LineNumber, e.LinePosition, e.Name, e.Text));
}
protected void handleUnknownAttribute(object sender, XmlAttributeEventArgs e)
{
log.write(
string.Format(
"XML_ERROR: Unknown Attribute at line {0} position {1} : {2}='{3}'",
e.LineNumber, e.LinePosition, e.Attr.Name, e.Attr.Value));
}
}
/// <summary>
/// xtractXmlReader provides bindings to the extract.xml
/// returned by the RIME server; as defined by:
/// $/Release X/Documentation/Technical/SCHEMA and
/// DTDs/nrw-rime-extract.xsd
/// </summary>
internal class ExtractXmlReader_XmlTextReaderXmlSerializerHybridImpl :
ExtractXmlReader
{
private Log log = Log.getInstance();
public rimeType read(string xmlFilename)
{
log.write(
string.Format(
"DEBUG: ExtractXmlReader_XmlTextReaderXmlSerializerHybridImpl." +
"read({0})",
xmlFilename));
using (XmlReader reader = XmlReader.Create(xmlFilename))
{
return read(reader);
}
}
public rimeType read(XmlReader reader)
{
rimeType result = new rimeType();
// a deserializer for featureClass, feature, etc, "doclets"
Dictionary<Type, XmlSerializer> serializers =
new Dictionary<Type, XmlSerializer>();
serializers.Add(typeof(featureClassType),
newSerializer(typeof(featureClassType)));
serializers.Add(typeof(featureType),
newSerializer(typeof(featureType)));
List<featureClassType> featureClasses = new List<featureClassType>();
List<featureType> features = new List<featureType>();
while (!reader.EOF)
{
if (reader.MoveToContent() != XmlNodeType.Element)
{
reader.Read(); // skip non-element-nodes and unknown-elements.
continue;
}
// skip junk nodes.
if (reader.Name.Equals("featureClass"))
{
using (
StringReader elementReader =
new StringReader(reader.ReadOuterXml()))
{
XmlSerializer deserializer =
serializers[typeof (featureClassType)];
featureClasses.Add(
(featureClassType)
deserializer.Deserialize(elementReader));
}
continue;
// ReadOuterXml advances the reader, so don't read again.
}
if (reader.Name.Equals("feature"))
{
using (
StringReader elementReader =
new StringReader(reader.ReadOuterXml()))
{
XmlSerializer deserializer =
serializers[typeof (featureType)];
features.Add(
(featureType)
deserializer.Deserialize(elementReader));
}
continue;
// ReadOuterXml advances the reader, so don't read again.
}
log.write(
"WARNING: unknown element '" + reader.Name +
"' was skipped during parsing.");
reader.Read(); // skip non-element-nodes and unknown-elements.
}
result.featureClasses = featureClasses.ToArray();
result.features = features.ToArray();
return result;
}
private XmlSerializer newSerializer(Type elementType)
{
XmlSerializer serializer = new XmlSerializer(elementType);
serializer.UnknownNode += new XmlNodeEventHandler(handleUnknownNode);
serializer.UnknownAttribute +=
new XmlAttributeEventHandler(handleUnknownAttribute);
return serializer;
}
protected void handleUnknownNode(object sender, XmlNodeEventArgs e)
{
log.write(
string.Format(
"XML_ERROR: Unknown Node at line {0} position {1} : {2}\t{3}",
e.LineNumber, e.LinePosition, e.Name, e.Text));
}
protected void handleUnknownAttribute(object sender, XmlAttributeEventArgs e)
{
log.write(
string.Format(
"XML_ERROR: Unknown Attribute at line {0} position {1} : {2}='{3}'",
e.LineNumber, e.LinePosition, e.Attr.Name, e.Attr.Value));
}
}
}
发布于 2009-03-24 07:23:49
总而言之,对于任何在google中找到这个帖子的人来说,答案都会更明显一些。
在XML2之前,XmlTextReader是标准.NET中内存效率最高的解析器(thanx Mitch;-)
.NET 2引入了XmlReader类,这个类更好,它是一个只向前元素迭代器(有点像StAX解析器)。(感谢Cerebrus;-)
孩子们,记住,任何XML实例都有可能超过500k,不要使用DOM!
干杯。基思。
https://stackoverflow.com/questions/676274
复制相似问题