Speeding up Addition in Arrays in Powershell

Note that the use of the += addition operator on an array in powershell can run /very/ slowly if you’re working with a large array. This is because under the hood an array is allocated a fixed chunk of memory and modifying the array requires creating a newly sized fixed chunk of memory and copying all of the data into the new memory.

https://docs.microsoft.com/en-us/powershell/scripting/dev-cross-plat/performance/script-authoring-considerations?view=powershell-7.2#array-addition

There are two ways of working around this. Either by using a c# type System.Collection.ArrayList or by using a neat powershell trick (below). If you’re only working on small arrays the overhead of this is going to be negligible and may not warrant rewriting to an arraylist.

[pastacode lang=”powershell” manual=”Slow%20Example%0A%0A%23%20an%20empty%20array%20is%20used%20as%20a%20bucket%20to%20collect%20results%3A%0A%0A%24bucket%C2%A0%3D%C2%A0%40()%0A%0A%0A%0A%0A%23%20use%20a%20stopwatch%20to%20measure%20performance%0A%0A%24stopwatch%C2%A0%3D%C2%A0%5BSystem.Diagnostics.Stopwatch%5D%3A%3AStartNew()%0A%0A%0A%0A%0A%23%20a%20loop%20does%20something%2C%20i.e.%20scans%20computers%2C%20processes%20database%20records%2C%0A%0A%23%20examines%20files%2C%20etc.%3A%0A%0A1..10000%C2%A0%7C%C2%A0Foreach-Object%C2%A0%7B%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%23%20result%20are%20added%20to%20the%20bucket%20using%20the%20%22%2B%3D%22%20operator%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%24bucket%C2%A0%2B%3D%C2%A0%22I%20am%20adding%C2%A0%24_%22%0A%0A%7D%0A%0A%0A%0A%0A%23%20all%20results%20end%20up%20in%20the%20array%3A%0A%0A%24report%C2%A0%3D%C2%A0’%7B0%7D%20elements%20collected%20in%20%7B1%3An1%7D%20seconds’%0A%0A%24report%C2%A0-f%C2%A0%24bucket.Count%2C%C2%A0%24stopwatch.Elapsed.TotalSeconds%0A%0A%0A%0A%0A10000%20elements%20collected%20in%205.2%20seconds%0A%0A%0A%0A%0AFast%20example%0A%0A%0A%0A%0A%23%20an%20empty%20ArrayList%20is%20used%20as%20a%20bucket%20to%20collect%20results%3A%0A%0A%24bucket%C2%A0%3D%C2%A0%5BSystem.Collections.ArrayList%5D%40()%0A%0A%0A%0A%0A%23%20use%20a%20stopwatch%20to%20measure%20performance%0A%0A%24stopwatch%C2%A0%3D%C2%A0%5BSystem.Diagnostics.Stopwatch%5D%3A%3AStartNew()%0A%0A%0A%0A%0A%23%20a%20loop%20does%20something%2C%20i.e.%20scans%20computers%2C%20processes%20database%20records%2C%0A%0A%23%20examines%20files%2C%20etc.%3A%0A%0A1..10000%C2%A0%7C%C2%A0Foreach-Object%C2%A0%7B%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%23%20result%20are%20added%20using%20the%20Add()%20method%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20%24null%C2%A0%3D%C2%A0%24bucket.Add(%22I%20am%20adding%C2%A0%24_%22)%0A%0A%7D%0A%0A%0A%0A%0A%23%20all%20results%20end%20up%20in%20the%20array%3A%0A%0A%24report%C2%A0%3D%C2%A0’%7B0%7D%20elements%20collected%20in%20%7B1%3An1%7D%20seconds’%0A%0A%24report%C2%A0-f%C2%A0%24bucket.Count%2C%C2%A0%24stopwatch.Elapsed.TotalSeconds%0A%0A%0A%0A%0A10000%20elements%20collected%20in%201.7%20seconds%0A%0A%0A%0A%0AEven%20faster%20example%0A%0A%23%20use%20a%20stopwatch%20to%20measure%20performance%0A%0A%24stopwatch%C2%A0%3D%C2%A0%5BSystem.Diagnostics.Stopwatch%5D%3A%3AStartNew()%0A%0A%0A%0A%0A%23%20let%20PowerShell%20handle%20object%20creation%0A%0A%24bucket%C2%A0%3D%C2%A01..10000%C2%A0%7C%C2%A0Foreach-Object%C2%A0%7B%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%23%20simply%20return%20the%20result%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%23%20PowerShell%20wraps%20all%20results%20in%20an%20array%20automatically%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%22I%20am%20adding%C2%A0%24_%22%0A%0A%7D%0A%0A%0A%0A%0A%23%20all%20results%20end%20up%20in%20the%20array%3A%0A%0A%24report%C2%A0%3D%C2%A0’%7B0%7D%20elements%20collected%20in%20%7B1%3An1%7D%20seconds’%0A%0A%24report%C2%A0-f%C2%A0%24bucket.Count%2C%C2%A0%24stopwatch.Elapsed.TotalSeconds%0A%0A%0A%0A%0A10000%20elements%20collected%20in%200.8%20seconds” message=”” highlight=”” provider=”manual”/]

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...