sqlldr性能调优测试(r3第5天)

接着续上次提到的sqlldr的性能问题,加载一个表数据400多万条记录,竟然用了6个多小时。最后大家争论不休的时候,我发现应该是网络的问题。 http://blog.itpub.net/23718752/viewspace-1182534/ 今天客户IT的同事把网络做了调整,他们就想看看到底改进有多大。 下面是测试的一些记录。

àoriginal logs for issue table, loading around 6 hours. 
Total logical records skipped:          0 
Total logical records read:       4096786 
Total logical records rejected:         0 
Total logical records discarded:        0 
Run began on Wed Jun 11 08:52:55 2014 
Run ended on Wed Jun 11 14:57:40 2014 
Elapsed time was:     06:04:44.05 
CPU time was:         00:00:38.18 
àafter network enhanced, loading time has reduced to around 4 mins. 
Total logical records skipped:          0 
Total logical records read:       4096786 
Total logical records rejected:         0 
Total logical records discarded:        0 
Run began on Mon Jun 16 12:58:01 2014 
Run ended on Mon Jun 16 13:02:15 2014 
Elapsed time was:     00:04:13.28 
CPU time was:         00:00:30.00 
àconsidering impact from index, I loaded data again, keep table structure,index exactly the same. 
Total logical records skipped:          0 
Total logical records read:       4096786 
Total logical records rejected:         0 
Total logical records discarded:        0 
Run began on Mon Jun 16 13:09:34 2014 
Run ended on Mon Jun 16 13:16:37 2014 
Elapsed time was:     00:07:02.61 
CPU time was:         00:00:31.23 
àtune sqlldr command with below options, keep table structure,index exactly the same. 
parallel=true bindsize=7500000 readsize=7500000 streamsize=7500000 rows=50000


Total logical records skipped:          0

Total logical records read:       4096786 
Total logical records rejected:         0 
Total logical records discarded:        0 
Run began on Mon Jun 16 13:21:13 2014 
Run ended on Mon Jun 16 13:24:19 2014 
Elapsed time was:     00:03:05.75 
CPU time was:         00:00:41.27

所以综上所述,在排查了cpu,io,缓存,网络的因素后,还可以使用一些选项来优化sqlldr,在某种程度上得到提升。

原文发布于微信公众号 - 杨建荣的学习笔记(jianrong-notes)

原文发表时间:2014-06-16

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏一个会写诗的程序员的博客

Spring Reactor 项目核心库Reactor Core

Non-Blocking Reactive Streams Foundation for the JVM both implementing a Reactiv...

2112
来自专栏大内老A

The .NET of Tomorrow

Ed Charbeneau(http://developer.telerik.com/featured/the-net-of-tomorrow/) Exciti...

30810
来自专栏闻道于事

js登录滑动验证,不滑动无法登陆

js的判断这里是根据滑块的位置进行判断,应该是用一个flag判断 <%@ page language="java" contentType="text/html...

6638
来自专栏杨龙飞前端

scrollto 到指定位置

2494
来自专栏魂祭心

原 canvas绘制clock

4024
来自专栏我和未来有约会

Kit 3D 更新

Kit3D is a 3D graphics engine written for Microsoft Silverlight. Kit3D was inita...

2506
来自专栏芋道源码1024

熔断器 Hystrix 源码解析 —— 断路器 HystrixCircuitBreaker

本文主要基于 Hystrix 1.5.X 版本 1. 概述 2. HystrixCircuitBreaker 3. HystrixCircuitBreaker....

5277
来自专栏陈仁松博客

ASP.NET Core 'Microsoft.Win32.Registry' 错误修复

今天在发布Asp.net Core应用到Azure的时候出现错误InvalidOperationException: Cannot find compilati...

4808
来自专栏我和未来有约会

Silverlight第三方控件专题

这里我收集整理了目前网上silverlight第三方控件的专题,若果有所遗漏请告知我一下。 名称 简介 截图 telerik 商 RadC...

3955
来自专栏Ceph对象存储方案

Luminous版本PG 分布调优

Luminous版本开始新增的balancer模块在PG分布优化方面效果非常明显,操作也非常简便,强烈推荐各位在集群上线之前进行这一操作,能够极大的提升整个集群...

3095

扫码关注云+社区