Go strings.Contains()比Python3慢2倍？
Go strings.Contains()比Python3慢2倍？
提问于 2018-07-31 19:01:48
我正在将一个文本模式扫描器从Python3转换到Go1.10，但我很惊讶它实际上慢了2倍。经过分析，罪魁祸首在strings.Contains()中。请参阅下面的简单基准测试。我错过什么了吗？你能推荐一种更快的Go模式搜索算法，在这种情况下会执行得更好吗？我不担心启动时间，同样的模式也会被用来扫描数百万个文件。
Py3基准测试：
import time
import re
RUNS = 10000
if __name__ == '__main__':
with open('data.php') as fh:
testString = fh.read()
def do():
return "576ad4f370014dfb1d0f17b0e6855f22" in testString
start = time.time()
for i in range(RUNS):
_ = do()
duration = time.time() - start
print("Python: %.2fs" % duration)Go1.10基准：
package main
import (
"fmt"
"io/ioutil"
"log"
"strings"
"time"
)
const (
runs = 10000
)
func main() {
fname := "data.php"
testdata := readFile(fname)
needle := "576ad4f370014dfb1d0f17b0e6855f22"
start := time.Now()
for i := 0; i < runs; i++ {
_ = strings.Contains(testdata, needle)
}
duration := time.Now().Sub(start)
fmt.Printf("Go: %.2fs\n", duration.Seconds())
}
func readFile(fname string) string {
data, err := ioutil.ReadFile(fname)
if err != nil {
log.Fatal(err)
}
return string(data)
}data.php是一个528KB的文件，可以是found here。
输出：
Go: 1.98s
Python: 0.84s回答 2
Stack Overflow用户
回答已采纳
发布于 2018-08-05 03:47:30
我用我在Wikipedia上找到的各种字符串搜索实现进行了更多的基准测试，例如：
- https://github.com/cloudflare/ahocorasick
- https://github.com/cubicdaiya/bms
- https://github.com/kkdai/kmp
- https://github.com/paddie/gokmp
- https://github.com/hillu/go-yara (Yara似乎在幕后实现了阿霍和科拉西克)。
基准测试结果(code here)：
BenchmarkStringsContains-4 10000 208055 ns/op
BenchmarkBMSSearch-4 1000 1856732 ns/op
BenchmarkPaddieKMP-4 2000 1069495 ns/op
BenchmarkKkdaiKMP-4 1000 1440147 ns/op
BenchmarkAhocorasick-4 2000 935885 ns/op
BenchmarkYara-4 1000 1237887 ns/op然后，我对我的实际用例进行了基准测试，针对本机(strings.Contains和regexp)和基于C的Yara实现，针对500KB的非匹配文件测试了大约1100个签名(100个正则表达式，1000个文字)：
BenchmarkScanNative-4 2 824328504 ns/op
BenchmarkScanYara-4 300 5338861 ns/op尽管Go中的C调用被认为是昂贵的，但在这些“繁重”的操作中，利润是显著的。侧面观察:匹配1100个签名而不是1个，Yara只需要5倍的CPU时间。
Stack Overflow用户
发布于 2018-08-01 02:39:07
为什么Python 3 (24.79s)比Go (5.47s)慢4.5倍？你得到了什么结果？
Python
$ cat contains.py
import time
import re
RUNS = 10000
if __name__ == '__main__':
# The Complete Works of William Shakespeare by William Shakespeare
# http://www.gutenberg.org/files/100/100-0.txt
file = '/home/peter/shakespeare.100-0.txt' # 'data.php'
with open(file) as fh:
testString = fh.read()
def do():
return "Means to immure herself and not be seen." in testString
start = time.time()
for i in range(RUNS):
_ = do()
duration = time.time() - start
print("Python: %.2fs" % duration)
print(do())
$ python3 --version
Python 3.6.5
$ python3 contains.py
Python: 24.79s
True
$ Go
$ cat contains.go
package main
import (
"fmt"
"io/ioutil"
"log"
"strings"
"time"
)
const (
runs = 10000
)
func main() {
// The Complete Works of William Shakespeare by William Shakespeare
// http://www.gutenberg.org/files/100/100-0.txt
fname := `/home/peter/shakespeare.100-0.txt` // "data.php"
testdata := readFile(fname)
needle := "Means to immure herself and not be seen."
start := time.Now()
for i := 0; i < runs; i++ {
_ = strings.Contains(testdata, needle)
}
duration := time.Now().Sub(start)
fmt.Printf("Go: %.2fs\n", duration.Seconds())
fmt.Println(strings.Contains(testdata, needle))
fmt.Println(strings.Index(testdata, needle))
}
func readFile(fname string) string {
data, err := ioutil.ReadFile(fname)
if err != nil {
log.Fatal(err)
}
return string(data)
}
$ go version
go version devel +5332b5e75a Tue Jul 31 15:44:37 2018 +0000 linux/amd64
$ go run contains.go
Go: 5.47s
true
5837178
$ 页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/51611862
复制相关文章
相似问题