blocks|key|889099|text|哇。哈希图(字典)可能不是您要查找的结构。|type|unstyled|depth|inlineStyleRanges|offset|length|style|BOLD|entityRanges|data|889100|不使用字符串，而是尝试一种可以提供良好和快速散列的表示。或者你真的在存储字符串？如果是这样，请省略前一句中的“可能”。|889101|你能详细介绍一下你正在解决的问题吗？|889102|entityMap^0|9|C|0|0|1N|0|0|I|0^^$0|@$1|2|3|4|5|6|7|L|8|@$9|M|A|N|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|O|8|@$9|P|A|Q|B|C]]|D|@]|E|$]]|$1|H|3|I|5|6|7|R|8|@$9|S|A|T|B|C]]|D|@]|E|$]]|$1|J|3|-4|5|6|7|U|8|@]|D|@]|E|$]]]|K|$]]

Wow. A hashmap (dictionary) might not be the structure you're looking for.

Instead of using strings, try a representation that can give you good and fast hashing. Or are you really storing strings? If so, scracth the "might" in the previous sentence.

Could you give us details about the problem you're tackling?

blocks|key|889136|text|正如圣地亚哥·莱西卡所说，字典不是你要找的结构。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|889137|也许你应该试试Redis：http://redis.io。它是一个高级的键值存储。|offset|length|889138|有一个用于python+here的库。|889139|entityMap|0|LINK|mutability|MUTABLE|url|http://redis.io/|1|http://pypi.python.org/pypi/redis^0|0|D|F|0|0|C|4|1|0^^$0|@$1|2|3|4|5|6|7|R|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|S|8|@]|9|@$D|T|E|U|1|V]]|A|$]]|$1|F|3|G|5|6|7|W|8|@]|9|@$D|X|E|Y|1|Z]]|A|$]]|$1|H|3|-4|5|6|7|10|8|@]|9|@]|A|$]]]|I|$J|$5|K|L|M|A|$N|O]]|P|$5|K|L|M|A|$N|Q]]]]

As Santiago Lezica said, a dictionary is not the structure you're looking for.

Maybe you should try Redis : <a href="http://redis.io" rel="nofollow">http://redis.io</a> . It's a advanced key-value store.

There is a library for python <a href="http://pypi.python.org/pypi/redis" rel="nofollow">here</a>.

blocks|key|1046277|text|PyTables+http://www.pytables.org/moin它是用来存储大型数据集的。在您的例子中，一个字典=一个表|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1046278|entityMap|0|LINK|mutability|MUTABLE|url|http://www.pytables.org/moin^0|9|S|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

PyTables <a href="http://www.pytables.org/moin" rel="nofollow">http://www.pytables.org/moin</a>
It's made to store large datasets. On your case, one dictionary = one table

blocks|key|889192|text|从你的描述来看，听起来你还不如这样做：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|889193|set(names1).intersection(set(names2))|code-block|syntax|javascript|889194|对吗？|889195|无论哪种方式，听起来问题似乎是您的算法很慢，而不是Python的哈希表的实现。|889196|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|N|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|O|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|P|8|@]|9|@]|A|$]]|$1|K|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|L|$]]

From your description it sounds like you might as well do:

<pre><code>set(names1).intersection(set(names2))
</code></pre>

Right?

Either way, it sounds like the problem is that your algorithm is slow, rather than the implementation of Python's hashtables.

blocks|key|3204913|text|即使不使用类或方法调用，也要将代码放入一个函数中并调用该函数。Python的函数进行了高度优化，部分原因是它访问局部变量的速度比访问全局变量的速度快。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3204914|Python+wiki上的Python+Performance+Tips文章是关于这个主题的很好的读物。|offset|length|3204915|entityMap|0|LINK|mutability|MUTABLE|url|http://wiki.python.org/moin/PythonSpeed/PerformanceTips^0|0|D|N|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|O|8|@]|9|@$D|P|E|Q|1|R]]|A|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|A|$]]]|G|$H|$5|I|J|K|A|$L|M]]]]

Even if not using classes or method calls, put your code into a function and call that function. Python's functions are highly optimized partly because it accesses local variables faster than global variables.

The <a href="http://wiki.python.org/moin/PythonSpeed/PerformanceTips" rel="nofollow">Python Performance Tips</a> writeup on the Python wiki is a great read on this topic.

blocks|key|1046338|text|问题:这是可伸缩性问题吗？你有没有发现当你有两倍的数据时，代码的运行速度是原来的两倍？您是否可能正在耗尽物理内存并使用交换内存？|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1046339|每个100个字符的1000万个字符串是一个千兆字节。如果您有2组这样的内存，那么就是2+32，这接近32位WinXP进程的极限。|1046340|如果您还不知道这个问题的答案，我建议对不同大小(+10或2的幂)的数据库运行一个测试，看看性能曲线是否不连续。|1046341|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|H|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|I|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|J|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|G|$]]

Questions: Is this a scaling issue ? Are you finding that the code runs more than twice as slow when you have twice as much data? Is it possible that you are running out of physical memory and using swap memory ?

10 million strings of 100 characters each is a gigabyte. If you have 2 sets of those, that would be 2 gigabytes, which is close to the limit of a 32 bit WinXP process.

If you don't already know the answer to this question, I would recommend running a test with the database at various sizes ( powers of 10 or 2 ) and see if the performance curve has a discontinuity.

I have few python scripts where I am storing 5-10 million string key value pairs in a dictionary and I query this dictionary around 5-10 million times. I noticed that python dict is not performing very well. Is there any other implementation best suited for string keys.

Edit:

I am having two large lists of person names and I want to match them, so I take one of them as the reference list and try applying different heuristics on each name in second list to figure out if that exists in the first list. So, I have to query first list 2-3 times for every name in second list. Hope, this makes sense.

Python: Best Dictionary implementation

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我有几个python脚本，我在字典中存储了500万到1000万个字符串键值对，我在这个字典中查询了大约500万到1000万次。我注意到python dict的性能不是很好。有没有其他最适合字符串键的实现。编辑：我有两个很大的人名列表，我想要匹配它们，所以我将其中一个作为参考列表，并尝试对第二个列表中的每个名称应用不同的...

问Python:最佳字典实现
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Python:最佳字典实现EN