I'm in the process of writing a PowerShell script that will change text in a script (from " to ') and then append text to the beginning and ends of each line. The actual code of the script isn't important as I have that down pat.
The questions/concerns here are that I'm unable to successfully leverage this script in a file that contains 7,000,000 lines (7 million lines is just a small sample of what I'm going to need when this rolls out into production) without getting some form of a WriteError/IOException. In addition the script is unable to process anything greater than approximately a bit over 550,000 lines of text at a time so I need to produce source files of 500,000 lines of information or less.
Secondly, if you try to execute this script using the same 500,000 lines (or less) of text sample as noted above and then try to open the file in notepad during said script execution:
(get-content sourcefilename.txt) | foreach-object {add-content destinationfile.txt “Fififofum $_”}
The script will fail with numerous errors.
Perhaps I’m missing something but when a script is writing to a new file shouldn’t there be a way (out of the box) to lock any external process out until the script has finished writing to the new file or open the file in a state that would show the state of the destination file at the time of execution without disturbing the script process? I shouldn't have to write additional code or have to worry whether a script is going to not execute when setting up automation of this sort. This feature should be standard with any sort of scripting environment.
Finally, when executing a PowerShell script using the code above on a file 500,000 lines or under, there isn’t a guarantee that the script will fully execute without the following error:
add-content : The process cannot access the file '(absolute path location)\destinationfile.txt' because it is being used by another process. At line:1 char:47+ (get-content sourcefilename.txt) | foreach-object {add-content destinationfilename.txt " ...+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+ CategoryInfo : WriteError: ((absolute path location)\destinationfile.txt) [Add-Content], IOException+ FullyQualifiedErrorId : GetContentWriterIOError,Microsoft.PowerShell.Commands.AddContentCommand
Based on my tests, I can only conclude that even by loading the file into memory and/or adding the ‘ReadCount’ option to the command before script execution, there isn’t a guarantee that the script will fully execute without getting these errors.
When running a test with the same script noted above producing two output files, one that generated the error and one that didn't, the file that produced the error didn't write a line at all at what I'm assuming is the exact point of the error. The entire line is blank (this includes the text available in the source file of the script noted above). Again, this scenario is if I load the file into memory or not or if I use the 'ReadCount' tag or not.
Since I'm getting these errors and the importance of making sure that the script is indeed doing what I want it to, I need to spend the time manually examining each output file that generates these errors to make sure that all the lines are modified correctly.
I shouldn't have to do this now, should I?
If I'm missing something here, please let me know.
My questions/feature requests are:
- Is the proper place to post bugs/feature requests?
- Can you make future editions of PowerShell be able to handle files with more than 500,000 lines?
- Can you add some sort of feature that will lock out all other potential processes or interruptions while a PowerShell script is executing?
- Can you add a feature that can handle large amounts of text modification without WriteErrors as noted above?
If any further elaboration is needed on these scenarios, I am more than happy to oblige! Thanks for taking the time to read this!