blocks|key|1025440|text|这可以通过使用索引或嵌套的ifelse()语句跳过循环来更快地完成。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1025441|idx+<-+1:nrow(temp)
temp[,10]+<-+idx
idx1+<-+c(FALSE,+(temp[-nrow(temp),6]+==+temp[-1,6])+&+(temp[-nrow(temp),3]+==+temp[-1,3]))
temp[idx1,10]+<-+temp[idx1,9]+%2B+temp[which(idx1)-1,10]+
temp[!idx1,10]+<-+temp[!idx1,9]++++
temp[1,10]+<-+temp[1,9]
names(temp)[names(temp)+==+"V10"]+<-+"Kumm."|code-block|syntax|javascript|1025442|entityMap^0|D|8|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@$9|N|A|O|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|P|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|Q|8|@]|D|@]|E|$]]]|L|$]]

This could be made much faster by skipping the loops by using indexes or nested <code>ifelse()</code> statements.

<pre><code>idx &lt;- 1:nrow(temp)
temp[,10] &lt;- idx
idx1 &lt;- c(FALSE, (temp[-nrow(temp),6] == temp[-1,6]) &amp; (temp[-nrow(temp),3] == temp[-1,3]))
temp[idx1,10] &lt;- temp[idx1,9] + temp[which(idx1)-1,10] 
temp[!idx1,10] &lt;- temp[!idx1,9] 
temp[1,10] &lt;- temp[1,9]
names(temp)[names(temp) == "V10"] &lt;- "Kumm."
</code></pre>

blocks|key|1240150|text|在R中，您通常可以通过使用apply系列函数(在您的示例中，它可能是replicate)来加速循环处理。请看一下提供进度条的plyr包。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1240151|另一种选择是完全避免循环，并将其替换为矢量化算法。我不确定您到底在做什么，但是您可以一次将您的函数应用于所有行：|1240152|temp[1:nrow(temp),+10]+<-+temp[1:nrow(temp),+9]+%2B+temp[0:(nrow(temp)-1),+10]|code-block|syntax|javascript|1240153|这会快得多，然后你就可以用你的条件来筛选行了：|1240154|cond.i+<-+(temp[i,+6]+==+temp[i-1,+6])+&+(temp[i,+3]+==+temp[i-1,+3])
temp[cond.i,+10]+<-+temp[cond.i,+9]|1240155|矢量化算法需要更多的时间和思考问题，但有时你可以节省几个数量级的执行时间。|1240156|entityMap^0|D|5|Y|9|1Q|4|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|U|8|@$9|V|A|W|B|C]|$9|X|A|Y|B|C]|$9|Z|A|10|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|11|8|@]|D|@]|E|$]]|$1|H|3|I|5|J|7|12|8|@]|D|@]|E|$K|L]]|$1|M|3|N|5|6|7|13|8|@]|D|@]|E|$]]|$1|O|3|P|5|J|7|14|8|@]|D|@]|E|$K|L]]|$1|Q|3|R|5|6|7|15|8|@]|D|@]|E|$]]|$1|S|3|-4|5|6|7|16|8|@]|D|@]|E|$]]]|T|$]]

In R, you can often speed-up loop processing by using the <code>apply</code> family functions (in your case, it would probably be <code>replicate</code>). Have a look at the <code>plyr</code> package that provides progress bars.

Another option is to avoid loops altogether and replace them with vectorized arithmetics. I'm not sure exactly what you are doing, but you can probably apply your function to all rows at once:

<pre><code>temp[1:nrow(temp), 10] &lt;- temp[1:nrow(temp), 9] + temp[0:(nrow(temp)-1), 10]
</code></pre>

This will be much much faster, and then you can filter the rows with your condition:

<pre><code>cond.i &lt;- (temp[i, 6] == temp[i-1, 6]) &amp; (temp[i, 3] == temp[i-1, 3])
temp[cond.i, 10] &lt;- temp[cond.i, 9]
</code></pre>

Vectorized arithmetics requires more time and thinking about the problem, but then you can sometimes save several orders of magnitude in execution time.

blocks|key|1240205|text|如果你使用的是for循环，你很可能会把R当作C、Java或其他东西来编码。经过适当向量化的R代码速度非常快。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1240206|以这两个简单的代码为例，按顺序生成一个包含10,000个整数的列表：|1240207|第一个代码示例是如何使用传统的编码范例编写循环。需要28秒才能完成|1240208|system.time({
++++a+<-+NULL
++++for(i+in+1:1e5)a[i]+<-+i
})
+++user++system+elapsed+
++28.36++++0.07+++28.61+|code-block|syntax|javascript|1240209|您可以通过预先分配内存的简单操作获得近100倍的性能提升：|1240210|system.time({
++++a+<-+rep(1,+1e5)
++++for(i+in+1:1e5)a[i]+<-+i
})

+++user++system+elapsed+
+++0.30++++0.00++++0.29+|1240211|但是使用基本的R向量操作，使用冒号运算符:，这个操作几乎是瞬间的：|1240212|system.time(a+<-+1:1e5)

+++user++system+elapsed+
++++++0+++++++0+++++++0+|1240213|entityMap^0|7|3|0|0|0|0|0|0|K|1|0|0^^$0|@$1|2|3|4|5|6|7|Y|8|@$9|Z|A|10|B|C]]|D|@]|E|$]]|$1|F|3|G|5|6|7|11|8|@]|D|@]|E|$]]|$1|H|3|I|5|6|7|12|8|@]|D|@]|E|$]]|$1|J|3|K|5|L|7|13|8|@]|D|@]|E|$M|N]]|$1|O|3|P|5|6|7|14|8|@]|D|@]|E|$]]|$1|Q|3|R|5|L|7|15|8|@]|D|@]|E|$M|N]]|$1|S|3|T|5|6|7|16|8|@$9|17|A|18|B|C]]|D|@]|E|$]]|$1|U|3|V|5|L|7|19|8|@]|D|@]|E|$M|N]]|$1|W|3|-4|5|6|7|1A|8|@]|D|@]|E|$]]]|X|$]]

If you are using <code>for</code> loops, you are most likely coding R as if it was C or Java or something else. R code that is properly vectorised is extremely fast.

Take for example these two simple bits of code to generate a list of 10,000 integers in sequence:

The first code example is how one would code a loop using a traditional coding paradigm. It takes 28 seconds to complete

<pre><code>system.time({
 a &lt;- NULL
 for(i in 1:1e5)a[i] &lt;- i
})
 user system elapsed 
 28.36 0.07 28.61 
</code></pre>

You can get an almost 100-times improvement by the simple action of pre-allocating memory:

<pre><code>system.time({
 a &lt;- rep(1, 1e5)
 for(i in 1:1e5)a[i] &lt;- i
})

 user system elapsed 
 0.30 0.00 0.29 
</code></pre>

But using the base R vector operation using the colon operator <code>:</code> this operation is virtually instantaneous:

<pre><code>system.time(a &lt;- 1:1e5)

 user system elapsed 
 0 0 0 
</code></pre>

blocks|key|1240260|text|使用data.table进行处理是一个可行的选择：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1240261|n+<-+1000000
df+<-+as.data.frame(matrix(sample(1:10,+n*9,+TRUE),+n,+9))
colnames(df)+<-+paste("col",+1:9,+sep+=+"")

library(data.table)

dayloop2.dt+<-+function(df)+{
++dt+<-+data.table(df)
++dt[,+Kumm.+:=+{
++++res+<-+.I;
++++ifelse+(res+>+1,+++++++++++++
++++++ifelse+((col6+==+shift(col6,+fill+=+0))+&+(col3+==+shift(col3,+fill+=+0))+,+
++++++++res+<-+col9+%2B+shift(res)+++++++++++++++++++
++++++,+#+else
++++++++res+<-+col9+++++++++++++++++++++++++++++++++
++++++)
+++++,+#+else
++++++res+<-+col9
++++)
++}
++,]
++res+<-+data.frame(dt)
++return+(res)
}

res+<-+dayloop2.dt(df)

m+<-+microbenchmark(dayloop2.dt(df),+times+=+100)
#Unit:+milliseconds
#+++++++expr++++++min++++++++lq+++++mean+++median+++++++uq++++++max+neval
#dayloop2.dt(df)+436.4467+441.02076+578.7126+503.9874+575.9534+966.1042++++10|code-block|syntax|javascript|1240262|如果忽略条件过滤可能带来的收益，则速度非常快。显然，如果您可以在数据子集上进行计算，这将有所帮助。|1240263|entityMap^0|2|A|0|0|0^^$0|@$1|2|3|4|5|6|7|O|8|@$9|P|A|Q|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|R|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|S|8|@]|D|@]|E|$]]|$1|M|3|-4|5|6|7|T|8|@]|D|@]|E|$]]]|N|$]]

Processing with <code>data.table</code> is a viable option:

<pre><code>n &lt;- 1000000
df &lt;- as.data.frame(matrix(sample(1:10, n*9, TRUE), n, 9))
colnames(df) &lt;- paste("col", 1:9, sep = "")

library(data.table)

dayloop2.dt &lt;- function(df) {
 dt &lt;- data.table(df)
 dt[, Kumm. := {
 res &lt;- .I;
 ifelse (res &gt; 1, 
 ifelse ((col6 == shift(col6, fill = 0)) &amp; (col3 == shift(col3, fill = 0)) , 
 res &lt;- col9 + shift(res) 
 , # else
 res &lt;- col9 
 )
 , # else
 res &lt;- col9
 )
 }
 ,]
 res &lt;- data.frame(dt)
 return (res)
}

res &lt;- dayloop2.dt(df)

m &lt;- microbenchmark(dayloop2.dt(df), times = 100)
#Unit: milliseconds
# expr min lq mean median uq max neval
#dayloop2.dt(df) 436.4467 441.02076 578.7126 503.9874 575.9534 966.1042 10
</code></pre>

If you ignore the possible gains from conditions filtering, it is very fast. Obviously, if you can do the calculation on the subset of data, it helps.

blocks|key|1240308|text|我不喜欢重写代码...当然，ifelse和lapply是更好的选择，但有时很难使其适合。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1240309|我经常像使用df$var[i]这样的列表一样使用data.frames|offset|length|style|CODE|1240310|下面是一个虚构的例子：|1240311|nrow=function(x){+##required+as+I+use+nrow+at+times.
++if(class(x)=='list')+{
++++length(x[[names(x)[1]]])
++}else{
++++base::nrow(x)
++}
}

system.time({
++d=data.frame(seq=1:10000,r=rnorm(10000))
++d$foo=d$r
++d$seq=1:5
++mark=NA
++for(i+in+1:nrow(d)){
++++if(d$seq[i]==1)+mark=d$r[i]
++++d$foo[i]=mark
++}
})

system.time({
++d=data.frame(seq=1:10000,r=rnorm(10000))
++d$foo=d$r
++d$seq=1:5
++d=as.list(d)+#become+a+list
++mark=NA
++for(i+in+1:nrow(d)){
++++if(d$seq[i]==1)+mark=d$r[i]
++++d$foo[i]=mark
++}
++d=as.data.frame(d)+#revert+back+to+data.frame
})|code-block|syntax|javascript|1240312|data.frame版本：|1240313|+++user++system+elapsed+
+++0.53++++0.00++++0.53|1240314|列表版本：|1240315|+++user++system+elapsed+
+++0.04++++0.00++++0.03+|1240316|使用向量列表比使用data.frame快17倍。|1240317|对于为什么内部data.frames在这方面如此缓慢，有什么评论吗？有人会认为它们像列表一样运作……|1240318|要获得更快的代码，请使用class(d)='list'代替d=as.list(d)和class(d)='data.frame'|1240319|system.time({
++d=data.frame(seq=1:10000,r=rnorm(10000))
++d$foo=d$r
++d$seq=1:5
++class(d)='list'
++mark=NA
++for(i+in+1:nrow(d)){
++++if(d$seq[i]==1)+mark=d$r[i]
++++d$foo[i]=mark
++}
++class(d)='data.frame'
})
head(d)|1240320|entityMap^0|0|6|9|0|0|0|0|0|0|0|0|0|C|F|T|C|16|L|0|0^^$0|@$1|2|3|4|5|6|7|16|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|17|8|@$D|18|E|19|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|1A|8|@]|9|@]|A|$]]|$1|J|3|K|5|L|7|1B|8|@]|9|@]|A|$M|N]]|$1|O|3|P|5|6|7|1C|8|@]|9|@]|A|$]]|$1|Q|3|R|5|L|7|1D|8|@]|9|@]|A|$M|N]]|$1|S|3|T|5|6|7|1E|8|@]|9|@]|A|$]]|$1|U|3|V|5|L|7|1F|8|@]|9|@]|A|$M|N]]|$1|W|3|X|5|6|7|1G|8|@]|9|@]|A|$]]|$1|Y|3|Z|5|6|7|1H|8|@]|9|@]|A|$]]|$1|10|3|11|5|6|7|1I|8|@$D|1J|E|1K|F|G]|$D|1L|E|1M|F|G]|$D|1N|E|1O|F|G]]|9|@]|A|$]]|$1|12|3|13|5|L|7|1P|8|@]|9|@]|A|$M|N]]|$1|14|3|-4|5|6|7|1Q|8|@]|9|@]|A|$]]]|15|$]]

I dislike rewriting code... Also of course ifelse and lapply are better options but sometimes it is difficult to make that fit.

Frequently I use data.frames as one would use lists such as <code>df$var[i]</code>

Here is a made up example:

<pre><code>nrow=function(x){ ##required as I use nrow at times.
 if(class(x)=='list') {
 length(x[[names(x)[1]]])
 }else{
 base::nrow(x)
 }
}

system.time({
 d=data.frame(seq=1:10000,r=rnorm(10000))
 d$foo=d$r
 d$seq=1:5
 mark=NA
 for(i in 1:nrow(d)){
 if(d$seq[i]==1) mark=d$r[i]
 d$foo[i]=mark
 }
})

system.time({
 d=data.frame(seq=1:10000,r=rnorm(10000))
 d$foo=d$r
 d$seq=1:5
 d=as.list(d) #become a list
 mark=NA
 for(i in 1:nrow(d)){
 if(d$seq[i]==1) mark=d$r[i]
 d$foo[i]=mark
 }
 d=as.data.frame(d) #revert back to data.frame
})
</code></pre>

data.frame version:

<pre><code> user system elapsed 
 0.53 0.00 0.53
</code></pre>

list version:

<pre><code> user system elapsed 
 0.04 0.00 0.03 
</code></pre>

17x times faster to use a list of vectors than a data.frame.

Any comments on why internally data.frames are so slow in this regard? One would think they operate like lists...

For even faster code do this <code>class(d)='list'</code> instead of <code>d=as.list(d)</code> and <code>class(d)='data.frame'</code>

<pre><code>system.time({
 d=data.frame(seq=1:10000,r=rnorm(10000))
 d$foo=d$r
 d$seq=1:5
 class(d)='list'
 mark=NA
 for(i in 1:nrow(d)){
 if(d$seq[i]==1) mark=d$r[i]
 d$foo[i]=mark
 }
 class(d)='data.frame'
})
head(d)
</code></pre>

blocks|key|1240364|text|看看{purrr}中的accumulate()函数：|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1240365|dayloop_accumulate+<-+function(temp)+{
++temp+%25>%25
++++as_tibble()+%25>%25
+++++mutate(cond+=+c(FALSE,+(V6+==+lag(V6)+&+V3+==+lag(V3))[-1]))+%25>%25
++++mutate(V10+=+V9+%25>%25+
+++++++++++++purrr::accumulate2(.y+=+cond[-1],+.f+=+function(.i_1,+.i,+.y)+{
+++++++++++++++if(.y)+{
+++++++++++++++++.i_1+%2B+.i
+++++++++++++++}+else+{
+++++++++++++++++.i
+++++++++++++++}
+++++++++++++})+%25>%25+unlist())+%25>%25
++++select(-cond)
}|code-block|syntax|javascript|1240366|entityMap^0|2|7|B|C|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@$9|N|A|O|B|C]|$9|P|A|Q|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|R|8|@]|D|@]|E|$I|J]]|$1|K|3|-4|5|6|7|S|8|@]|D|@]|E|$]]]|L|$]]

Take a look at the <code>accumulate()</code> function from <code>{purrr}</code> :
<pre><code>dayloop_accumulate &lt;- function(temp) {
 temp %&gt;%
 as_tibble() %&gt;%
 mutate(cond = c(FALSE, (V6 == lag(V6) &amp; V3 == lag(V3))[-1])) %&gt;%
 mutate(V10 = V9 %&gt;% 
 purrr::accumulate2(.y = cond[-1], .f = function(.i_1, .i, .y) {
 if(.y) {
 .i_1 + .i
 } else {
 .i
 }
 }) %&gt;% unlist()) %&gt;%
 select(-cond)
}
</code></pre>

I have a big performance problem in R. I wrote a function that iterates over a <code>data.frame</code> object. It simply adds a new column to a <code>data.frame</code> and accumulates something. (simple operation). The <code>data.frame</code> has roughly 850K rows. My PC is still working (about 10h now) and I have no idea about the runtime.

<pre><code>dayloop2 &lt;- function(temp){
 for (i in 1:nrow(temp)){ 
 temp[i,10] &lt;- i
 if (i &gt; 1) { 
 if ((temp[i,6] == temp[i-1,6]) &amp; (temp[i,3] == temp[i-1,3])) { 
 temp[i,10] &lt;- temp[i,9] + temp[i-1,10] 
 } else {
 temp[i,10] &lt;- temp[i,9] 
 }
 } else {
 temp[i,10] &lt;- temp[i,9]
 }
 }
 names(temp)[names(temp) == "V10"] &lt;- "Kumm."
 return(temp)
}
</code></pre>

Any ideas how to speed up this operation?

Speed up the loop operation in R

Java

我在R中遇到了一个很大的性能问题。我写了一个遍历data.frame对象的函数。它只是向data.frame添加了一个新列，并累积了一些内容。(操作简单)。data.frame大约有850K行。我的电脑还在工作(大约10小时)，我对运行时间一无所知。dayloop2 <- function(temp){ for ...

问加速R中的循环操作
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问加速R中的循环操作EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问加速R中的循环操作
EN