PowerShell: Moving files based on extension

The following script works but is incredibly slow. Does anyone have a faster solution?

In short, I have a folder full (4M+) of recovered files. I would like to create a subfolder for each based on extension and move the file there. As I said, the following script is incredibly slow.

get-childitem -file | ForEach-Object -parallel {$ext=$_.Extension; mkdir $ext; mv $_ .\$ext\} -throttlelimit 32

It is slow regardless of whether it runs in parallel as written or without parallel.

Thanks!

UPDATE

This problem occurs in versions 7 both with and without parallel being enabled. Without parallel enable in version 7, it takes about 15 seconds per file.

I have just tried the script in version 5 (without parallel, obviously) and it is processing well in excess of 100+ files per second.

So, this problem is now merely academic. Running the script in version 5 is reasonable fast and should complete the 4M+ files in less than 15 minutes at current pace.

Why does it take versions 7 15-ish seconds per file for the following script:

get-childitem -file | ForEach-Object {$ext=$_.Extension; mkdir $ext; mv $_ .\$ext\}

3 Answers

Try to improve the speed by creating the folders first, then moving the files in one command per each extension:

$extensions = Get-Childitem | Select-Object -Unique @{label='ext';expression={$_.Extension.substring(1)}}
$extensions | foreach {New-Item -ItemType directory -Name $_.ext}
$extensions | foreach {mv "*.$($_.ext)" "$($_.ext)"}

I'd suggest you first create an array of all extensions found in the folder (without leading dot) and use robocopy to move the files while also creating the subdirectories:

$sourceFolder = $PSScriptRoot # specify the path where the files are now
# get a unique array of all file extensions found in the source folder
$extensions = (Get-ChildItem -Path $sourceFolder -File | Group-Object {$_.Extension.TrimStart(".")}).Name
# loop through the extension array and have robocopy do the copying for you
foreach ($ext in $extensions) { $targetDir = Join-Path -Path $sourceFolder -ChildPath $ext robocopy $sourceFolder $targetDir "*.$ext" /MOV /NFL /NDL /NJH /NJS /NC /NS /NP /R:0 /W:0 > $null
}

Your existing command might try to create a directory with each file and extension so if you have 1000 PDF documents in the folder, it runs the command 1000 times to create a .pdf subfolder.

With the -Unique parameter of the Select command, it only runs that command one time for each distinct or unique file extension value to create a folder with the dot extension (e.g. c:\ParentFolder\.pdf).

I'd just pipe all the "unique" file extension values to another loop right over to the New-Item command and use the -Force parameter with it. Try to avoid throwing another variable into the mix if possible.

The next part I would run the Get-ChildItem command and loop it right into the Move-Item command to move the file with the matching extension to the same dot extension named subfolder.

Get-ChildItem -File | % { Process { $_.Extension }} | Select -Unique | % { Process { New-Item $_ -ItemType Directory -Force }};
Get-ChildItem -File | % { Process { Move-Item $_ -Destination $_.Extension -Force }};

Supporting Resources

• ForEach-Object

  Standard Aliases for Foreach-Object: the '%' symbol, ForEach

• New-Item
• Move-Item