blocks|key|1736186|text|我使用了相同的方法，即为每条记录存储一个2列行，然后使用格式化的html转到一个txt文件，然后改为json，最后改为mongodb。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1736187|但由于您有频繁的操作，我建议使用cassandra、hbase和google等大型表实现，如accumulo和hypertable。|1736188|云数据可能是适合你的。|1736189|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|H|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|I|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|J|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|G|$]]

I used the same method that is, to store a 2 column row for every record, which I turned to a txt file with the formatted html which then we changed to json and finally to mongodb.

But since you have got frequent operations, I suggest cassandra, hbase and googles big table implementations like accumulo cloudata and hypertable.

Cloudata may be the right one for you.

blocks|key|1736227|text|在卡桑德拉，你可以有宽行，最多每一行2B列.如果这足以满足实体的累积列表项，则可以将整个实体的列表存储在一行中，然后将其全部检索到一起。使用cassandra的“复合列”，您可以按顺序存储每个列表的元素，您可以在需要时删除单个列(列表项)，当插入时只需插入一列.|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1736228|就像这样：(!)|1736229|++++++%7Clist_1_Id+:+item1Id+%7Clist_1_Id+:+item2Id+%7C+list_2_Id+:+item1Id+%7C...%7C+list_n_Id+:+item3Id+%7C
entity%7C+++++item1Value+++++%7C+++++item2Value+++++%7C+++++item1Value++++++%7C...%7C+++++item3Value++++++%7C|code-block|syntax|javascript|1736230|因此，实际上您处理的是列(=items)，而不是列表。这样你的工作就容易多了。取决于您的列表大小局外人使用分割entiti的行为多行..。就像这样：(!)|1736231|+++++++++++++++++%7C++item1Id+++%7C++item2Id+++%7C++item3Id+++%7C++item4Id+++%7C...
entiId_list_1_Id+%7C+item1Value+%7C+item2Value+%7C+item3Value+%7C+item4Value+%7C...
+++++++++++++++++%7C++item1Id+++%7C++item2Id+++%7C++item3Id+++%7C++item4Id+++%7C...
entiId_list_2_Id+%7C+item1Value+%7C+item2Value+%7C+item3Value+%7C+item4Value+%7C...
...|1736232|您可以将itemValue放在列名中，并保留列值为空以减少大小.例如，只需执行以下操作即可插入一个新项://列如果有任何插入到entityListentityIditemId+=项目值中，则按其id排序；或//列按值insert排序到entityListentityIditemvalue+=+nothing；删除:+delete:+entityList中的delete+=‘d’和listId='o‘和itemId='n'；|1736233|或者通过您的应用程序，您可以通过使用像赫克托这样的富客户端来实现.|offset|length|1736234|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/hector-client/hector^0|0|0|0|0|0|0|J|3|0|0^^$0|@$1|2|3|4|5|6|7|10|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|11|8|@]|9|@]|A|$]]|$1|D|3|E|5|F|7|12|8|@]|9|@]|A|$G|H]]|$1|I|3|J|5|6|7|13|8|@]|9|@]|A|$]]|$1|K|3|L|5|F|7|14|8|@]|9|@]|A|$G|H]]|$1|M|3|N|5|6|7|15|8|@]|9|@]|A|$]]|$1|O|3|P|5|6|7|16|8|@]|9|@$Q|17|R|18|1|19]]|A|$]]|$1|S|3|-4|5|6|7|1A|8|@]|9|@]|A|$]]]|T|$U|$5|V|W|X|A|$Y|Z]]]]

In cassandra you can have wide rows, up to 2B columns per row... if that's enough for an entity's cumulative lists' item, you can store whole entity's lists in a single row then retrieve it all together.
with cassandra's "composite column" you can store elements of each list sequentially and ordered and you can delete a single column(a list item) when you want, and when you have an insertion you just need to insert a column...

something like this: (!)

<pre><code> |list_1_Id : item1Id |list_1_Id : item2Id | list_2_Id : item1Id |...| list_n_Id : item3Id |
entity| item1Value | item2Value | item1Value |...| item3Value |
</code></pre>

so practically you deal with columns(=items) rather than lists... and it makes your work much easier.
depends on your lists size cosider using spliting entiti's row to multiple rows...
something like this: (!)

<pre><code> | item1Id | item2Id | item3Id | item4Id |...
entiId_list_1_Id | item1Value | item2Value | item3Value | item4Value |...
 | item1Id | item2Id | item3Id | item4Id |...
entiId_list_2_Id | item1Value | item2Value | item3Value | item4Value |...
...
</code></pre>

and you can put itemValue in column name and leave column value empty to reduce size...
for example you can insert a new item by simply doing:
 //columns are sorted by their id if they have any 
 insert into entityList[entityId][listId][itemId] = item value;
or
 //columns are sorted by their value
 insert into entityList[entityId][listId][itemvalue] = nothing;
and delete:
 delete from entityList where entityId='d' and listId='o' and itemId='n';

or via you application you can do it by using a rich client like <a href="https://github.com/hector-client/hector" rel="nofollow">Hector</a>...

blocks|key|1736286|text|我推荐SSDB(https://github.com/ideawu/ssdb)，一种Google网络包装器。SSDB用于存储收集数据，如list、map、zset(排序集)。你可以这样使用它：|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1736287|ssdb->hset(listId,+listItem1);
ssdb->hset(listId,+listItem2);
ssdb->hset(listId,+listItem3);
...
list+=+ssdb->hscan(listId,+100);
//+now+list+=+[listItem1,+listItem2,+listItem3,+...]|code-block|syntax|javascript|1736288|一个映射中的项目数仅限于硬盘大小。另一个解决方案是Redis，但是Redis将所有数据存储到内存中(比如不超过30+is+)，因此它可能不适合您的项目。|1736289|SSDB支持C%2B%2B、PHP、Python、Lua和更多客户端。|1736290|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/ideawu/ssdb^0|8|U|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@]|9|@$A|V|B|W|1|X]]|C|$]]|$1|D|3|E|5|F|7|Y|8|@]|9|@]|C|$G|H]]|$1|I|3|J|5|6|7|Z|8|@]|9|@]|C|$]]|$1|K|3|L|5|6|7|10|8|@]|9|@]|C|$]]|$1|M|3|-4|5|6|7|11|8|@]|9|@]|C|$]]]|N|$O|$5|P|Q|R|C|$S|T]]]]

I would recommend SSDB(<a href="https://github.com/ideawu/ssdb" rel="nofollow">https://github.com/ideawu/ssdb</a>), a Google leveldb network wrapper. SSDB is designed to store collection data, such as list, map, zset(sorted set). You can use it like this way:

<pre><code>ssdb-&gt;hset(listId, listItem1);
ssdb-&gt;hset(listId, listItem2);
ssdb-&gt;hset(listId, listItem3);
...
list = ssdb-&gt;hscan(listId, 100);
// now list = [listItem1, listItem2, listItem3, ...]
</code></pre>

The number of items in one map is only limited to the size of hard disk. Another solution is Redis, but Redis stores all data into memory(say no more than 30GB), so it probably won't fit your project.

C++, PHP, Python, Java, Lua, and more clients are supported by SSDB.

blocks|key|660403|text|Cassandra支持存储集合/地图/列表。如果您的查询总是拖垮整个事物，那么它们是处理这类事情的一个非常简单的方法。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|660404|集合+http://cassandra.apache.org/doc/cql3/CQL.html#collections|offset|length|660405|如果您的列表与用户绑定，则可以在用户行/分区上生成不同的列，然后对多个列表的查询将非常快速，因为对于给定用户而言，它们都位于同一个分区中。|660406|entityMap|0|LINK|mutability|MUTABLE|url|http://www.datastax.com/dev/blog/cql3_collections|1|http://cassandra.apache.org/doc/cql3/CQL.html#collections^0|0|0|2|0|3|1L|1|0|0^^$0|@$1|2|3|4|5|6|7|R|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|S|8|@]|9|@$D|T|E|U|1|V]|$D|W|E|X|1|Y]]|A|$]]|$1|F|3|G|5|6|7|Z|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|10|8|@]|9|@]|A|$]]]|I|$J|$5|K|L|M|A|$N|O]]|P|$5|K|L|M|A|$N|Q]]]]

Cassandra has native support for storing sets/maps/lists. If your queries will always be pulling the whole thing down, then they are a very easy way to deal with this type of thing.

<a href="http://www.datastax.com/dev/blog/cql3_collections" rel="nofollow">http://www.datastax.com/dev/blog/cql3_collections</a>
<a href="http://cassandra.apache.org/doc/cql3/CQL.html#collections" rel="nofollow">http://cassandra.apache.org/doc/cql3/CQL.html#collections</a>

If your lists are tied to a user, you can make the different columns on the users row/partition, and then queries for the multiple lists will be fast, as they will all be in the same partition for a given user.

blocks|key|660417|text|Cassandra可以很好地用于这样的用例。为返回的数据集/查询创建任意数量的Column族。Cassandra对非规范化数据或像1:m，m:m关系这样的集合效果最好.|type|unstyled|depth|inlineStyleRanges|entityRanges|data|660418|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

Cassandra can be used very well for such use cases. Create as many Columnfamilies as you want for the returned data sets/queries. Cassandra works best with de-normalized data or sets like 1:m, m:m relations.

blocks|key|660524|text|正如您所指出的，解决方案必须是可执行的和可扩展的:我建议您使用Redis和它的列表数据结构和O(1)插入和O+(N+)取(考虑到从列表中获取最后一个元素)，并使用一些散列算法水平缩放它。我不知道你要存储多少数据，有多少台机器可用，但从性能上看，这绝对是最好的选择，因为没有什么比内存访问速度更快了。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|660525|如果数据量很大，而且无法将其全部保存在RAM中，那么Cassandra就可以完成按时间排序的列表存储工作，就像上面提到的那样，使用分区策略来存储列表更合适。|660526|还有一个想法:您说读取性能必须是最大的，一旦用户登录，您将需要为这个用户获取上百个列表。为什么不为每个用户准备一个列表呢？这样就会有更多的写入，但是读取将被优化，因为您只需要从一个列表中获取最新的条目。我不确定这是否适合你的任务，只是一个想法。:)|660527|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|H|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|I|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|J|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|G|$]]

As you pointed out the solution must be performant and scaleable: I'd suggest you to use Redis with it's LIST data structure and O(1) inserts and O(N) fetches (N - elements to fetch, considering you're fetching last ones from lists) and scale it horizontally with some hashing algorithm. I don't know what amount of data you are going to store and how many machines are available, but definitely it will be the best choice performance-wise, since nothing beats memory access speed.

If the amount of data is huge and you can't keep it all in RAM then Cassandra can do the job - storing lists ordered by time is a nice fit for it even better with partition strategy as Zanson mentioned above.

One more thought: you said read performance must be max, and once user logs in you will need to fetch hundred of lists for this user. Why not to prepare a single list for each user? That way there will be more writes but the read will be optimized since you will need to fetch only latest entries from one list. I'm not sure if that fits your task, just a thought. :)

blocks|key|3056731|text|我知道您不想考虑关系数据库，但我认为对于这种简单的情况，也有一个带有关系数据库的可伸缩解决方案。主要的好处是您不需要维护一个单独的数据库系统。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3056732|为了获得可伸缩性，所有NoSQL解决方案将将您的数据分布在多个节点上。您可以在应用程序代码中这样做，将数据分散到多个关系数据库中。为了保持负载平衡，您可能需要偶尔移动数据，但只需为每个N个列表生成一个新的数据库就足够了。|3056733|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

I know you didn't want to consider relational databases, but I think for this simple situation, there is also a scalable solution with relational database. The main benefit would be that you don't need to maintain a separate database system.

To gain scalability, all NoSQL solutions will distribute your data across multiple nodes. You can do this in your application code, spreading your data out across multiple relational databases. To keep the load balanced, you may need to move data occasionally, but it may be sufficient to simply spawn a new database for every N lists.

For a social network site, I need to store frequently modified lists for each entity(&amp; millions of such entities) which are:

<blockquote>
 <ul>
 <li>frequently appended to </li>
 <li>frequently read </li>
 <li>sometimes reduced </li>
 <li>lists are keyed by primary key</li>
 </ul>
</blockquote>

I'm already storing some other type of data in an RDBMS. I know that I could store those lists in an RDBMS as a many to many relationship like this way: Create a table <code>listItems</code> with two columns <code>listId</code> &amp; <code>listItem</code> &amp; to generate any particular list, just do a <code>SELECT</code> query for all records <code>WHERE listId = x</code>. But storing lists this way in an RDBMS is not very ideal when high scalability is concerned. Instead I would like to store prepared lists in a natural way, so that retrieval performance is maximized. Because I need to fetch around hundred of such lists for a user, whenever I user does login &amp; view a page.

So how do I solve this ? What kind of database should be used for this data, probably the one that provide adding variable no of columns to keyed by a primary key, the ones like Cassandra ?

How to store frequently modified lists in database in a natural way so that they are just ready to read?

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

对于社交网站，我需要为每个实体(数百万这样的实体)存储经常修改的列表，这些实体是：频繁地附加到常读有时减少列表按主键。我已经在RDBMS中存储了一些其他类型的数据。我知道我可以将这些列表存储在RDBMS中，就像这样:创建一个包含两个列的表listItems & listId & listItem &要生成任何特定的列表...

问如何以一种自然的方式将频繁修改的列表存储在数据库中，以便它们能够随时阅读？
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何以一种自然的方式将频繁修改的列表存储在数据库中，以便它们能够随时阅读？EN