1

I want to write some random numbers into an ascii output file. I generate the numbers with numpy, so the numbers are stored in numpy.array

import numpy as np
random1=np.random.uniform(-1.2,1.2,7e6)
random2=...
random3=...

All three array are of the same size. I used standard file output, but this is really slow. Just about 8000 lines per 30 min. This may because I loop over three large arrays though.

fout1 = open("output.dat","w")

for i in range(len(random1)):
  fout1.write(str(random1[i])+"\t"+ str(random2[i])+"\t"+ str(random3[i])+"\n")
fout1.close()

I also just used print str(random1[i])+"\t"+ str(random2[i])+"\t"+ str(random3[i]) and dumped everything in a file usind shell ./myprog.py > output.dat which seems a bit faster but still I am not satisfied with the output speed.

Any recommendations are really welcome.

4

2 に答える 2

4

やってみました

random = np.vstack((random1, random2, random3)).T
random.savetxt("output.dat", delimiter="\t")
于 2013-02-11T09:26:12.103 に答える
0

disk ioは、実行している最もコストのかかる操作だと思います。これに対処するために、ループバッファごとにすべての行を書き込む代わりに、独自のバッファを作成して、1つの大きなブロックに書き込むことができます。次に、これを試して、最も有益なバッファサイズを確認します

于 2013-02-11T09:24:59.630 に答える