blocks|key|3136823|text|您可能会考虑使用Cube。它不是一个Django应用程序，但它有很多很好的特性，Django可以很容易地与它通信。而且，它也很可爱。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|3136824|​|3136825|📷|atomic|3136826|3136827|你可以让你的Django应用程序在事件发生时将事件发送到MongoDB中。这种系统分离将防止Django应用程序上的任何额外压力。|3136828|entityMap|0|LINK|mutability|MUTABLE|url|http://square.github.com/cube/|1|IMAGE|IMMUTABLE|imageUrl|https://ask.qcloudimg.com/http-save/yehe-900000/a22374dac3770ad47d159df3ab4735eb.png|imageAlt^0|8|4|0|0|0|0|1|1|0|0|0^^$0|@$1|2|3|4|5|6|7|Z|8|@]|9|@$A|10|B|11|1|12]]|C|$]]|$1|D|3|E|5|6|7|13|8|@]|9|@]|C|$]]|$1|F|3|G|5|H|7|14|8|@]|9|@$A|15|B|16|1|17]]|C|$]]|$1|I|3|E|5|6|7|18|8|@]|9|@]|C|$]]|$1|J|3|K|5|6|7|19|8|@]|9|@]|C|$]]|$1|L|3|-4|5|6|7|1A|8|@]|9|@]|C|$]]]|M|$N|$5|O|P|Q|C|$R|S]]|T|$5|U|P|V|C|$W|X|Y|-4]]]]

You might want to consider <a href="http://square.github.com/cube/" rel="noreferrer">Cube</a>. It is not a Django app, but it has a lot of nice features baked in, and Django can communicate to it easily. Also, it is lovely.

<img src="https://i.stack.imgur.com/3sUvq.png" alt="Cube screenshot">

You could have you Django app just blast off events into MongoDB when the occur. This separation of systems would prevent any additional strain on your Django app.

blocks|key|248319|text|对不起，Mridang+Agarwalla，有时你的问题会浮现在我的脑海中。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|248320|我认为保持，OLAP和OLTP两个数据库同步的方法是最新的，对OLTP的影响很小。|248321|在2002年，我成功地使用了这种技术来解决类似的问题。它的工作原理如下：|248322|248323|您可以为每个事实表编写触发器。当事实数据被修改时，触发器在一个表中插入一行来反映这个事件(idEvent+%2B+update+%7C+delete+%7C+insert+%2B事实表的外键)。OLAP+|ordered-list-item|248324|低优先级守护进程执行无限循环，对于每个循环迭代，您将从表中‘弹出’10个事件，并使用这些新信息更新|248325|数据库。|248326|您可以优化守护进程的行为，例如，如果表没有新事件，守护进程可以休眠15秒。|248327|在我的escenario中，只有事实表有触发器。如果事实表引用的数据不在OLAP数据库中，则我在此时创建该数据(OLTP和OLAP具有不同的模式)。|248328|如果您分析您的数据库，您可以找到数百个表，但只有几个表是真正的事实表。|248329|好吧，我知道这只是你答案的一部分。您问题的第二部分谈到了用于分析data的强大工具。我不能向你推荐任何开源产品(因为我没有开源分析工具的经验)。我在前台使用过Microsoft+Analysis+Services+%2B+Tableau软件。这是一个非常好的解决方案，但我不知道它是否与您的filosofy相匹配。对于数据挖掘，您有KNIME+(Konstanz+Information+Miner)，它是一个用户友好的、全面的开源数据集成、处理、分析和探索平台(但您需要一个以前的ETL任务)。|offset|length|style|BOLD|248330|请把你的项目的消息发给我，我很感兴趣。我有一个django学生出勤解决方案，我想添加分析功能。|248331|entityMap|0|LINK|mutability|MUTABLE|url|http://www.knime.org/^0|0|0|0|0|0|0|0|0|0|0|11|5T|4K|5|0|0|0|1B|0^^$0|@$1|2|3|4|5|6|7|19|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1A|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|1B|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|1C|8|@]|9|@]|A|$]]|$1|G|3|H|5|I|7|1D|8|@]|9|@]|A|$]]|$1|J|3|K|5|I|7|1E|8|@]|9|@]|A|$]]|$1|L|3|M|5|6|7|1F|8|@]|9|@]|A|$]]|$1|N|3|O|5|6|7|1G|8|@]|9|@]|A|$]]|$1|P|3|Q|5|6|7|1H|8|@]|9|@]|A|$]]|$1|R|3|S|5|6|7|1I|8|@]|9|@]|A|$]]|$1|T|3|U|5|6|7|1J|8|@$V|1K|W|1L|X|Y]]|9|@$V|1M|W|1N|1|1O]]|A|$]]|$1|Z|3|10|5|6|7|1P|8|@$V|1Q|W|1R|X|Y]]|9|@]|A|$]]|$1|11|3|-4|5|6|7|1S|8|@]|9|@]|A|$]]]|12|$13|$5|14|15|16|A|$17|18]]]]

Sorry Mridang Agarwalla, some times your question come back to mind ...

I thought the way to keep in sync both databases, OLAP and OLTP, up to date with low impact in OLTP.

In 2002 I successfully employed this technique for a similar issue. It works as follows:

<ol>
<li>You write a trigger for each fact table. When fact data is modified trigger inserts a row in a table that reflects this event (idEvent + update | delete | insert + foreign key to fact table). </li>
<li>A low priority daemon do a infinite loop, for each loop iteration you 'pop' 10 events from table and update OLAP database with this new information. </li>
</ol>

You can optimize daemon behavior, for example, if table don't has new events daemon can sleep for 15 seconds.

In my escenario only fact tables have trigger. If fact table reference data that no is in OLAP database I created the data at this time (OLTP and OLAP has different schema). 

If you analize your database you can find hundred of tables but only few tables are really fact tables.

Well, I konw this is only a partial answer of your answer. Second part of your question talk about a power tool to analize data. I can't suggest to you any open source product (because I have not experience with open source analysis tools). I have worked with Microsoft Analysis Services + Tableau software in frontend. This is a very nice solution but I don't know if it matches with your filosofy. For data minning you have <a href="http://www.knime.org/" rel="nofollow">KNIME</a> (Konstanz Information Miner) that is a user-friendly and comprehensive open-source data integration, processing, analysis, and exploration platform (but you need a previus ETL task). 

Please, sent to me news about your project, I'm very interested. I have a django student attendance solution and I want to add analysis functionality.

I need to build in a analytics (reporting, charting &amp; graphing) system into my Django application. In an ideal world I could just query my Postgres DB and get the data I want but when the amount of data in the DB goes through the roof, I'll hit performance bottlenecks and other issues like index hell.

I'm wondering if you could point me in a right direction to implement this:

<ul>
<li>Is this a good scenario to use a NoSQL DB like (CouchDB, MongoDB, Redis) and query the data from that?</li>
<li>Since Postgres and Django have no OLAP/MDX support should I go along with a star-schema in a different databse and query that?</li>
</ul>

I'm looking to avoid two things:

<ul>
<li>I don't want to query my actual DB for analytics as it might take a huge performance hit.</li>
<li>I'd like to keep my analytics as up to date as possible i.e. I'd like to incrementally update my data warehouse to have a the latest data. Every time, there's a CRUD operation on my transactional DB, I'd like to update the data warehouse.</li>
</ul>

This is yet another scenario that I haven't worked with and am trying to understand the quickest and best way to accomplish.

I hope I've been verbose enough. If not, I'dd gladly explain more.

Thanks everyone

<hr>

After digging around the web and using the knowledge I have, I've come to this solution:

Use the Postgres to store the relational data. On Every CRUD operation, call the analytics code to do the calculations on the data and store the data in a NoSQL DB like Redis/CouchDB.

Looking at this good comparison of the NoSQL DB's (http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis), I think Redis or CouchDB fits in just fine. Targeted for analytics.

I could store calculated Analytics in Redis/CouchDB and update them incrementally when my source data changes.

Is this a good solution?

Data Mining in a Django/Postgres application

数据仓库

数据存储

数据集成

数据挖掘

数据库

MongoDB

Django

Redis

NoSQL

OLAP

OLTP

我需要在我的Django应用程序中内置一个分析(报告，图表和绘图)系统。在理想情况下，我可以只查询Postgres数据库并获得我想要的数据，但当数据库中的数据量达到顶峰时，我会遇到性能瓶颈和其他问题，比如索引地狱。我想知道您是否可以为我指出一个正确的方向来实现这一点：这是使用NoSQL DB (CouchDB，Mong...

问Django/Postgres应用程序中的数据挖掘
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Django/Postgres应用程序中的数据挖掘EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Django/Postgres应用程序中的数据挖掘
EN