blocks|key|3317722|text|您的传输速度可能不会受到Java的限制。相反，我会怀疑(没有特定的顺序)|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3317723|从数据库到磁盘传输的速度|3317724|ordered-list-item|3317725|3317726|+of+transfer+to+the+disk|3317727|3317728|如果您读取完整的数据集，然后将其写出到磁盘，那么这将花费更长的时间，因为JVM将必须分配内存，并且db区域/磁盘写入将按顺序发生。相反，我会为你从数据库中进行的每一次读操作写出缓冲的写入器，这样操作就更接近于并发操作(我不知道你是不是在这样做)|3317729|entityMap^0|0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|O|8|@]|9|@]|A|$]]|$1|D|3|-4|5|E|7|P|8|@]|9|@]|A|$]]|$1|F|3|-4|5|E|7|Q|8|@]|9|@]|A|$]]|$1|G|3|H|5|E|7|R|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|S|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|T|8|@]|9|@]|A|$]]|$1|L|3|-4|5|6|7|U|8|@]|9|@]|A|$]]]|M|$]]

Your transfer speed is likely not to be limited by Java. Instead I would suspect (in no particular order)

<ol>
<li>the speed of transfer from the database</li>
<li>the speed of transfer to the disk</li>
</ol>

If you read the complete dataset and then write it out to disk, then that will take longer, since the JVM will have to allocate memory, and the db rea/disk write will happen sequentially. Instead I would write out to the buffered writer for every read that you make from the db, and so the operation will be closer to a concurrent one (I don't know if you're doing that or not)

blocks|key|3104925|text|您可以尝试删除BufferedWriter并直接使用FileWriter。在现代系统中，您很有可能只是在写入驱动器的高速缓存。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3104926|我写175MB+(400万个字符串)需要4-5秒的时间--这是在运行Windows+XP的双核2.4+the戴尔电脑上完成的，日立硬盘容量为80+XP、7200-RPM。|3104927|您能区分出有多少时间是记录检索，有多少是文件写入吗？|3104928|import+java.io.BufferedWriter;
import+java.io.File;
import+java.io.FileWriter;
import+java.io.IOException;
import+java.io.Writer;
import+java.util.ArrayList;
import+java.util.List;

public+class+FileWritingPerfTest+{
++++

private+static+final+int+ITERATIONS+=+5;
private+static+final+double+MEG+=+(Math.pow(1024,+2));
private+static+final+int+RECORD_COUNT+=+4000000;
private+static+final+String+RECORD+=+"Help+I+am+trapped+in+a+fortune+cookie+factory\n";
private+static+final+int+RECSIZE+=+RECORD.getBytes().length;

public+static+void+main(String[]+args)+throws+Exception+{
++++List<String>+records+=+new+ArrayList<String>(RECORD_COUNT);
++++int+size+=+0;
++++for+(int+i+=+0;+i+<+RECORD_COUNT;+i%2B%2B)+{
++++++++records.add(RECORD);
++++++++size+%2B=+RECSIZE;
++++}
++++System.out.println(records.size()+%2B+"+'records'");
++++System.out.println(size+/+MEG+%2B+"+MB");
++++
++++for+(int+i+=+0;+i+<+ITERATIONS;+i%2B%2B)+{
++++++++System.out.println("\nIteration+"+%2B+i);
++++++++
++++++++writeRaw(records);
++++++++writeBuffered(records,+8192);
++++++++writeBuffered(records,+(int)+MEG);
++++++++writeBuffered(records,+4+*+(int)+MEG);
++++}
}

private+static+void+writeRaw(List<String>+records)+throws+IOException+{
++++File+file+=+File.createTempFile("foo",+".txt");
++++try+{
++++++++FileWriter+writer+=+new+FileWriter(file);
++++++++System.out.print("Writing+raw...+");
++++++++write(records,+writer);
++++}+finally+{
++++++++//+comment+this+out+if+you+want+to+inspect+the+files+afterward
++++++++file.delete();
++++}
}

private+static+void+writeBuffered(List<String>+records,+int+bufSize)+throws+IOException+{
++++File+file+=+File.createTempFile("foo",+".txt");
++++try+{
++++++++FileWriter+writer+=+new+FileWriter(file);
++++++++BufferedWriter+bufferedWriter+=+new+BufferedWriter(writer,+bufSize);
++++
++++++++System.out.print("Writing+buffered+(buffer+size:+"+%2B+bufSize+%2B+")...+");
++++++++write(records,+bufferedWriter);
++++}+finally+{
++++++++//+comment+this+out+if+you+want+to+inspect+the+files+afterward
++++++++file.delete();
++++}
}

private+static+void+write(List<String>+records,+Writer+writer)+throws+IOException+{
++++long+start+=+System.currentTimeMillis();
++++for+(String+record:+records)+{
++++++++writer.write(record);
++++}
++++//+writer.flush();+//+close()+should+take+care+of+this
++++writer.close();+
++++long+end+=+System.currentTimeMillis();
++++System.out.println((end+-+start)+/+1000f+%2B+"+seconds");
}
}|code-block|syntax|javascript|3104929|entityMap^0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|N|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|O|8|@]|9|@]|A|$]]|$1|F|3|G|5|H|7|P|8|@]|9|@]|A|$I|J]]|$1|K|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|L|$]]

You might try removing the BufferedWriter and just using the FileWriter directly. On a modern system there's a good chance you're just writing to the drive's cache memory anyway.
It takes me in the range of 4-5 seconds to write 175MB (4 million strings) -- this is on a dual-core 2.4GHz Dell running Windows XP with an 80GB, 7200-RPM Hitachi disk.
Can you isolate how much of the time is record retrieval and how much is file writing?
<pre><code>import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.util.ArrayList;
import java.util.List;

public class FileWritingPerfTest {
 

private static final int ITERATIONS = 5;
private static final double MEG = (Math.pow(1024, 2));
private static final int RECORD_COUNT = 4000000;
private static final String RECORD = &quot;Help I am trapped in a fortune cookie factory\n&quot;;
private static final int RECSIZE = RECORD.getBytes().length;

public static void main(String[] args) throws Exception {
 List&lt;String&gt; records = new ArrayList&lt;String&gt;(RECORD_COUNT);
 int size = 0;
 for (int i = 0; i &lt; RECORD_COUNT; i++) {
 records.add(RECORD);
 size += RECSIZE;
 }
 System.out.println(records.size() + &quot; 'records'&quot;);
 System.out.println(size / MEG + &quot; MB&quot;);
 
 for (int i = 0; i &lt; ITERATIONS; i++) {
 System.out.println(&quot;\nIteration &quot; + i);
 
 writeRaw(records);
 writeBuffered(records, 8192);
 writeBuffered(records, (int) MEG);
 writeBuffered(records, 4 * (int) MEG);
 }
}

private static void writeRaw(List&lt;String&gt; records) throws IOException {
 File file = File.createTempFile(&quot;foo&quot;, &quot;.txt&quot;);
 try {
 FileWriter writer = new FileWriter(file);
 System.out.print(&quot;Writing raw... &quot;);
 write(records, writer);
 } finally {
 // comment this out if you want to inspect the files afterward
 file.delete();
 }
}

private static void writeBuffered(List&lt;String&gt; records, int bufSize) throws IOException {
 File file = File.createTempFile(&quot;foo&quot;, &quot;.txt&quot;);
 try {
 FileWriter writer = new FileWriter(file);
 BufferedWriter bufferedWriter = new BufferedWriter(writer, bufSize);
 
 System.out.print(&quot;Writing buffered (buffer size: &quot; + bufSize + &quot;)... &quot;);
 write(records, bufferedWriter);
 } finally {
 // comment this out if you want to inspect the files afterward
 file.delete();
 }
}

private static void write(List&lt;String&gt; records, Writer writer) throws IOException {
 long start = System.currentTimeMillis();
 for (String record: records) {
 writer.write(record);
 }
 // writer.flush(); // close() should take care of this
 writer.close(); 
 long end = System.currentTimeMillis();
 System.out.println((end - start) / 1000f + &quot; seconds&quot;);
}
}
</code></pre>

blocks|key|3104948|text|对于这些从DB读取的大容量数据，您可能需要调优语句的获取大小。这可能会节省大量到DB的往返时间。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3104949|http://download.oracle.com/javase/1.5.0/docs/api/java/sql/Statement.html#setFetchSize%2528int%2529|offset|length|3104950|entityMap|0|LINK|mutability|MUTABLE|url|http://download.oracle.com/javase/1.5.0/docs/api/java/sql/Statement.html#setFetchSize%2528int%2529^0|0|0|2M|0|0^^$0|@$1|2|3|4|5|6|7|N|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|O|8|@]|9|@$D|P|E|Q|1|R]]|A|$]]|$1|F|3|-4|5|6|7|S|8|@]|9|@]|A|$]]]|G|$H|$5|I|J|K|A|$L|M]]]]

For these bulky reads from DB you may want to tune your Statement's fetch size. It might save a lot of roundtrips to DB.

<a href="http://download.oracle.com/javase/1.5.0/docs/api/java/sql/Statement.html#setFetchSize%28int%29" rel="nofollow noreferrer">http://download.oracle.com/javase/1.5.0/docs/api/java/sql/Statement.html#setFetchSize%28int%29</a>

blocks|key|341053|text|尝试内存映射文件(在我的m/c，core+2+duo，2.5+my中写入174MB需要300m/s)：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|341054|byte[]+buffer+=+"Help+I+am+trapped+in+a+fortune+cookie+factory\n".getBytes();
int+number_of_lines+=+400000;

FileChannel+rwChannel+=+new+RandomAccessFile("textfile.txt",+"rw").getChannel();
ByteBuffer+wrBuf+=+rwChannel.map(FileChannel.MapMode.READ_WRITE,+0,+buffer.length+*+number_of_lines);
for+(int+i+=+0;+i+<+number_of_lines;+i%2B%2B)
{
++++wrBuf.put(buffer);
}
rwChannel.close();|code-block|syntax|javascript|341055|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

try memory mapped files (takes 300 m/s to write 174MB in my m/c, core 2 duo, 2.5GB RAM) :

<pre><code>byte[] buffer = "Help I am trapped in a fortune cookie factory\n".getBytes();
int number_of_lines = 400000;

FileChannel rwChannel = new RandomAccessFile("textfile.txt", "rw").getChannel();
ByteBuffer wrBuf = rwChannel.map(FileChannel.MapMode.READ_WRITE, 0, buffer.length * number_of_lines);
for (int i = 0; i &lt; number_of_lines; i++)
{
 wrBuf.put(buffer);
}
rwChannel.close();
</code></pre>

blocks|key|3105069|text|这仅仅是为了统计：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3105070|该计算机是配备新固态硬盘的旧戴尔产品|3105071|CPU:+Intel+Pentium+D+2,8+Ghz|3105072|固态硬盘:爱国者Inferno+120+SSD固态硬盘|3105073|4000000+'records'
175.47607421875+MB

Iteration+0
Writing+raw...+3.547+seconds
Writing+buffered+(buffer+size:+8192)...+2.625+seconds
Writing+buffered+(buffer+size:+1048576)...+2.203+seconds
Writing+buffered+(buffer+size:+4194304)...+2.312+seconds

Iteration+1
Writing+raw...+2.922+seconds
Writing+buffered+(buffer+size:+8192)...+2.406+seconds
Writing+buffered+(buffer+size:+1048576)...+2.015+seconds
Writing+buffered+(buffer+size:+4194304)...+2.282+seconds

Iteration+2
Writing+raw...+2.828+seconds
Writing+buffered+(buffer+size:+8192)...+2.109+seconds
Writing+buffered+(buffer+size:+1048576)...+2.078+seconds
Writing+buffered+(buffer+size:+4194304)...+2.015+seconds

Iteration+3
Writing+raw...+3.187+seconds
Writing+buffered+(buffer+size:+8192)...+2.109+seconds
Writing+buffered+(buffer+size:+1048576)...+2.094+seconds
Writing+buffered+(buffer+size:+4194304)...+2.031+seconds

Iteration+4
Writing+raw...+3.093+seconds
Writing+buffered+(buffer+size:+8192)...+2.141+seconds
Writing+buffered+(buffer+size:+1048576)...+2.063+seconds
Writing+buffered+(buffer+size:+4194304)...+2.016+seconds|code-block|syntax|javascript|3105074|正如我们所看到的，原始方法比缓冲方法更慢。|3105075|entityMap^0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|R|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|S|8|@]|9|@]|A|$]]|$1|F|3|G|5|6|7|T|8|@]|9|@]|A|$]]|$1|H|3|I|5|J|7|U|8|@]|9|@]|A|$K|L]]|$1|M|3|N|5|6|7|V|8|@]|9|@]|A|$]]|$1|O|3|-4|5|6|7|W|8|@]|9|@]|A|$]]]|P|$]]

Only for the sake of statistics: 

The machine is old Dell with new SSD

CPU: Intel Pentium D 2,8 Ghz

SSD: Patriot Inferno 120GB SSD

<pre><code>4000000 'records'
175.47607421875 MB

Iteration 0
Writing raw... 3.547 seconds
Writing buffered (buffer size: 8192)... 2.625 seconds
Writing buffered (buffer size: 1048576)... 2.203 seconds
Writing buffered (buffer size: 4194304)... 2.312 seconds

Iteration 1
Writing raw... 2.922 seconds
Writing buffered (buffer size: 8192)... 2.406 seconds
Writing buffered (buffer size: 1048576)... 2.015 seconds
Writing buffered (buffer size: 4194304)... 2.282 seconds

Iteration 2
Writing raw... 2.828 seconds
Writing buffered (buffer size: 8192)... 2.109 seconds
Writing buffered (buffer size: 1048576)... 2.078 seconds
Writing buffered (buffer size: 4194304)... 2.015 seconds

Iteration 3
Writing raw... 3.187 seconds
Writing buffered (buffer size: 8192)... 2.109 seconds
Writing buffered (buffer size: 1048576)... 2.094 seconds
Writing buffered (buffer size: 4194304)... 2.031 seconds

Iteration 4
Writing raw... 3.093 seconds
Writing buffered (buffer size: 8192)... 2.141 seconds
Writing buffered (buffer size: 1048576)... 2.063 seconds
Writing buffered (buffer size: 4194304)... 2.016 seconds
</code></pre>

As we can see the raw method is slower the buffered.

blocks|key|337560|text|​|type|unstyled|depth|inlineStyleRanges|entityRanges|data|337561|package+all.is.well;
import+java.io.IOException;
import+java.io.RandomAccessFile;
import+java.util.concurrent.ExecutorService;
import+java.util.concurrent.Executors;
import+junit.framework.TestCase;

/**
+*+@author+Naresh+Bhabat
+*+
Following++implementation+helps+to+deal+with+extra+large+files+in+java.
This+program+is+tested+for+dealing+with+2GB+input+file.
There+are+some+points+where+extra+logic+can+be+added+in+future.


Pleasenote:+if+we+want+to+deal+with+binary+input+file,+then+instead+of+reading+line,we+need+to+read+bytes+from+read+file+object.



It+uses+random+access+file,which+is+almost+like+streaming+API.


+*+****************************************
Notes+regarding+executor+framework+and+its+readings.
Please+note+:ExecutorService+executor+=+Executors.newFixedThreadPool(10);

+*++	+++for+10+threads:Total+time+required+for+reading+and+writing+the+text+in
+*+++++++++:seconds+349.317
+*+
+*+++++++++For+100:Total+time+required+for+reading+the+text+and+writing+++:+seconds+464.042
+*+
+*+++++++++For+1000+:+Total+time+required+for+reading+and+writing+text+:466.538+
+*+++++++++For+10000++Total+time+required+for+reading+and+writing+in+seconds+479.701
+*
+*+
+*/
public+class+DealWithHugeRecordsinFile+extends+TestCase+{

	static+final+String+FILEPATH+=+"C:\\springbatch\\bigfile1.txt.txt";
	static+final+String+FILEPATH_WRITE+=+"C:\\springbatch\\writinghere.txt";
	static+volatile+RandomAccessFile+fileToWrite;
	static+volatile+RandomAccessFile+file;
	static+volatile+String+fileContentsIter;
	static+volatile+int+position+=+0;

	public+static+void+main(String[]+args)+throws+IOException,+InterruptedException+{
		long+currentTimeMillis+=+System.currentTimeMillis();

		try+{
			fileToWrite+=+new+RandomAccessFile(FILEPATH_WRITE,+"rw");//for+random+write,independent+of+thread+obstacles+
			file+=+new+RandomAccessFile(FILEPATH,+"r");//for+random+read,independent+of+thread+obstacles+
			seriouslyReadProcessAndWriteAsynch();

		}+catch+(IOException+e)+{
			//+TODO+Auto-generated+catch+block
			e.printStackTrace();
		}
		Thread+currentThread+=+Thread.currentThread();
		System.out.println(currentThread.getName());
		long+currentTimeMillis2+=+System.currentTimeMillis();
		double+time_seconds+=+(currentTimeMillis2+-+currentTimeMillis)+/+1000.0;
		System.out.println("Total+time+required+for+reading+the+text+in+seconds+"+%2B+time_seconds);

	}

	/**
	+*+@throws+IOException
	+*+Something++asynchronously+serious
	+*/
	public+static+void+seriouslyReadProcessAndWriteAsynch()+throws+IOException+{
		ExecutorService+executor+=+Executors.newFixedThreadPool(10);//pls+see+for+explanation+in+comments+section+of+the+class
		while+(true)+{
			String+readLine+=+file.readLine();
			if+(readLine+==+null)+{
				break;
			}
			Runnable+genuineWorker+=+new+Runnable()+{
				@Override
				public+void+run()+{
					//+do+hard+processing+here+in+this+thread,i+have+consumed
					//+some+time+and+eat+some+exception+in+write+method.
					writeToFile(FILEPATH_WRITE,+readLine);
					//+System.out.println("+:"+%2B
					//+Thread.currentThread().getName());

				}
			};
			executor.execute(genuineWorker);
		}
		executor.shutdown();
		while+(!executor.isTerminated())+{
		}
		System.out.println("Finished+all+threads");
		file.close();
		fileToWrite.close();
	}

	/**
	+*+@param+filePath
	+*+@param+data
	+*+@param+position
	+*/
	private+static+void+writeToFile(String+filePath,+String+data)+{
		try+{
			//+fileToWrite.seek(position);
			data+=+"\n"+%2B+data;
			if+(!data.contains("Randomization"))+{
				return;
			}
			System.out.println("Let+us+do+something+time+consuming+to+make+this+thread+busy"%2B(position%2B%2B)+%2B+"+++:"+%2B+data);
			System.out.println("Lets+consume+through+this+loop");
			int+i=1000;
			while(i>0){
			
				i--;
			}
			fileToWrite.write(data.getBytes());
			throw+new+Exception();
		}+catch+(Exception+exception)+{
			System.out.println("exception+was+thrown+but+still+we+are+able+to+proceeed+further"
					%2B+"+\n+This+can+be+used+for+marking+failure+of+the+records");
			//exception.printStackTrace();

		}

	}
}|code-block|syntax|javascript|337562|337563|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|J|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|K|8|@]|9|@]|A|$E|F]]|$1|G|3|4|5|6|7|L|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|M|8|@]|9|@]|A|$]]]|I|$]]

<div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>package all.is.well;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import junit.framework.TestCase;

/**
 * @author Naresh Bhabat
 * 
Following implementation helps to deal with extra large files in java.
This program is tested for dealing with 2GB input file.
There are some points where extra logic can be added in future.


Pleasenote: if we want to deal with binary input file, then instead of reading line,we need to read bytes from read file object.



It uses random access file,which is almost like streaming API.


 * ****************************************
Notes regarding executor framework and its readings.
Please note :ExecutorService executor = Executors.newFixedThreadPool(10);

 * 	 for 10 threads:Total time required for reading and writing the text in
 * :seconds 349.317
 * 
 * For 100:Total time required for reading the text and writing : seconds 464.042
 * 
 * For 1000 : Total time required for reading and writing text :466.538 
 * For 10000 Total time required for reading and writing in seconds 479.701
 *
 * 
 */
public class DealWithHugeRecordsinFile extends TestCase {

	static final String FILEPATH = "C:\\springbatch\\bigfile1.txt.txt";
	static final String FILEPATH_WRITE = "C:\\springbatch\\writinghere.txt";
	static volatile RandomAccessFile fileToWrite;
	static volatile RandomAccessFile file;
	static volatile String fileContentsIter;
	static volatile int position = 0;

	public static void main(String[] args) throws IOException, InterruptedException {
		long currentTimeMillis = System.currentTimeMillis();

		try {
			fileToWrite = new RandomAccessFile(FILEPATH_WRITE, "rw");//for random write,independent of thread obstacles 
			file = new RandomAccessFile(FILEPATH, "r");//for random read,independent of thread obstacles 
			seriouslyReadProcessAndWriteAsynch();

		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		Thread currentThread = Thread.currentThread();
		System.out.println(currentThread.getName());
		long currentTimeMillis2 = System.currentTimeMillis();
		double time_seconds = (currentTimeMillis2 - currentTimeMillis) / 1000.0;
		System.out.println("Total time required for reading the text in seconds " + time_seconds);

	}

	/**
	 * @throws IOException
	 * Something asynchronously serious
	 */
	public static void seriouslyReadProcessAndWriteAsynch() throws IOException {
		ExecutorService executor = Executors.newFixedThreadPool(10);//pls see for explanation in comments section of the class
		while (true) {
			String readLine = file.readLine();
			if (readLine == null) {
				break;
			}
			Runnable genuineWorker = new Runnable() {
				@Override
				public void run() {
					// do hard processing here in this thread,i have consumed
					// some time and eat some exception in write method.
					writeToFile(FILEPATH_WRITE, readLine);
					// System.out.println(" :" +
					// Thread.currentThread().getName());

				}
			};
			executor.execute(genuineWorker);
		}
		executor.shutdown();
		while (!executor.isTerminated()) {
		}
		System.out.println("Finished all threads");
		file.close();
		fileToWrite.close();
	}

	/**
	 * @param filePath
	 * @param data
	 * @param position
	 */
	private static void writeToFile(String filePath, String data) {
		try {
			// fileToWrite.seek(position);
			data = "\n" + data;
			if (!data.contains("Randomization")) {
				return;
			}
			System.out.println("Let us do something time consuming to make this thread busy"+(position++) + " :" + data);
			System.out.println("Lets consume through this loop");
			int i=1000;
			while(i&gt;0){
			
				i--;
			}
			fileToWrite.write(data.getBytes());
			throw new Exception();
		} catch (Exception exception) {
			System.out.println("exception was thrown but still we are able to proceeed further"
					+ " \n This can be used for marking failure of the records");
			//exception.printStackTrace();

		}

	}
}</code></pre>
</div>
</div>

blocks|key|3317900|text|对于那些想要缩短检索记录和转储到文件中的时间(即不处理记录)的人，可以将这些记录附加到StringBuffer中，而不是将它们放入ArrayList中。应用toSring()函数获取单个字符串，并立即将其写入文件。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|3317901|对我来说，检索时间从22秒减少到17秒。|3317902|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

For those who want to improve the time for retrieval of records and dump into the file (i.e no processing on records), instead of putting them into an ArrayList, append those records into a StringBuffer. Apply toSring() function to get a single String and write it into the file at once. 

For me, the retrieval time reduced from 22 seconds to 17 seconds.

I have to write huge data in text[csv] file. I used BufferedWriter to write the data and it took around 40 secs to write 174 mb of data. Is this the fastest speed java can offer?

<pre><code>bufferedWriter = new BufferedWriter ( new FileWriter ( "fileName.csv" ) );
</code></pre>

Note: These 40 secs include the time of iterating and fetching the records from resultset as well. :) . 174 mb is for 400000 rows in resultset.

Fastest way to write huge data in text file Java

往返时间

内存映射

数据库

Windows

Java

我必须在textcsv文件中写入大量数据。我使用BufferedWriter来写数据，写174MB的数据花了大约40秒。这是java能提供的最快速度吗？bufferedWriter = new BufferedWriter ( new FileWriter ( "fileName.csv" ) );注意：这40秒还包括...

问在Java文本文件中写入大数据的最快方法
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问在Java文本文件中写入大数据的最快方法EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问在Java文本文件中写入大数据的最快方法
EN