So I did end up dividing the large text file into 26*26*26 blocks of files based on starting three characters, and it increased the speed to about 2 Kb /Second over a four hour period. As before the first hour was very fast presumably because of the lack of corresponding unscubscribies in the unsubscribe list.
The system resources consumed which initially were minimal increased to a huge number as shown below, and I think indicate that the file connections are not being closed properly, even though I use the StreamWriter.Dipose() method to close all file streams that are open.
In addition to the RAM consumed is constantly growing which is worrying me for the sake of program and whether its eventually going to crash.
The resources are now at 822,000 KB.
.....
No comments:
Post a Comment