使用Haskell，我如何处理大量的XML？
使用Haskell，我如何处理大量的XML？
提问于 2010-02-19 06:30:12
我一直在探索Stack Overflow data dumps，到目前为止，我利用了友好的XML和使用正则表达式“解析”的优势。我尝试使用各种Haskell XML库按文档顺序查找特定用户的第一篇帖子，但都遇到了令人讨厌的麻烦。
import Control.Monad
import Text.HTML.TagSoup
userid = "83805"
main = do
posts <- liftM parseTags (readFile "posts.xml")
print $ head $ map (fromAttrib "Id") $
filter (~== ("<row OwnerUserId=" ++ userid ++ ">"))
postsimport Text.XML.HXT.Arrow
import Text.XML.HXT.XPath
userid = "83805"
main = do
runX $ readDoc "posts.xml" >>> posts >>> arr head
where
readDoc = readDocument [ (a_tagsoup, v_1)
, (a_parse_xml, v_1)
, (a_remove_whitespace, v_1)
, (a_issue_warnings, v_0)
, (a_trace, v_1)
]
posts :: ArrowXml a => a XmlTree String
posts = getXPathTrees byUserId >>>
getAttrValue "Id"
where byUserId = "/posts/row/@OwnerUserId='" ++ userid ++ "'"import Control.Monad
import Control.Monad.Error
import Control.Monad.Trans.Maybe
import Data.Either
import Data.Maybe
import Text.XML.Light
userid = "83805"
main = do
[posts,votes] <- forM ["posts", "votes"] $
liftM parseXML . readFile . (++ ".xml")
let ps = elemNamed "posts" posts
putStrLn $ maybe "<not present>" show
$ filterElement (byUser userid) ps
elemNamed :: String -> [Content] -> Element
elemNamed name = head . filter ((==name).qName.elName) . onlyElems
byUser :: String -> Element -> Bool
byUser id e = maybe False (==id) (findAttr creator e)
where creator = QName "OwnerUserId" Nothing Nothing我哪里错了？使用Haskell处理大量XML文档的正确方式是什么？
回答 6
Stack Overflow用户
发布于 2010-02-19 07:36:40
我注意到在所有这些情况下，您都在执行字符串IO。如果您希望高效地处理大量文本，则绝对必须使用Data.Text或Data.Bytestring(.Lazy)，例如String == Char，这对于非常大的平面文件来说并不合适。
这意味着您将需要使用支持字节串的Haskell XML库。这里提供了几十个xml库：http://hackage.haskell.org/packages/archive/pkg-list.html#cat:xml
我不确定哪个支持字节串，但这就是你要找的条件。
Stack Overflow用户
发布于 2010-02-23 04:02:34
下面是一个使用hexpat的示例
{-# LANGUAGE PatternGuards #-}
module Main where
import Text.XML.Expat.SAX
import qualified Data.ByteString.Lazy as B
userid = "83805"
main :: IO ()
main = B.readFile "posts.xml" >>= print . earliest
where earliest :: B.ByteString -> SAXEvent String String
earliest = head . filter (ownedBy userid) . parse opts
opts = ParserOptions Nothing Nothing
ownedBy :: String -> SAXEvent String String -> Bool
ownedBy uid (StartElement "row" as)
| Just ouid <- lookup "OwnerUserId" as = ouid == uid
| otherwise = False
ownedBy _ _ = FalseownedBy的定义有点笨拙。也许是视图模式：
{-# LANGUAGE ViewPatterns #-}
module Main where
import Text.XML.Expat.SAX
import qualified Data.ByteString.Lazy as B
userid = "83805"
main :: IO ()
main = B.readFile "posts.xml" >>= print . earliest
where earliest :: B.ByteString -> SAXEvent String String
earliest = head . filter (ownedBy userid) . parse opts
opts = ParserOptions Nothing Nothing
ownedBy :: String -> SAXEvent String String -> Bool
ownedBy uid (ownerUserId -> Just ouid) = uid == ouid
ownedBy _ _ = False
ownerUserId :: SAXEvent String String -> Maybe String
ownerUserId (StartElement "row" as) = lookup "OwnerUserId" as
ownerUserId _ = NothingStack Overflow用户
发布于 2012-10-18 20:05:17
你可以试试我的fast-tagsoup库。它是tagsoup的简单替代品，解析速度为20-200MB/秒。
tagsoup包的问题是，即使你使用文本或ByteString接口，它也能在内部处理字符串。fast-tagsoup使用高性能的低级解析与严格的ByteStrings一起工作，同时仍然返回惰性标签列表作为输出。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/2292729
复制相关文章
相似问题