Microsoft cloud engineer - SharePoint, Office 365, Azure, DotNet, Angular, JavaScript.
Microsoft cloud engineer - SharePoint, Office 365, Azure, DotNet, Angular, JavaScript.

DataTable in PowerShell for crazy fast filters

Recently I was working with a 60,000 row CSV and applying several filters to find matching rows.   This took between 1.0-1.8 seconds per row.   While OK in small scale (few hundred) the script run duration grew to several hours.    There had to be a better way.

Enter [System.Data.DataTable]

Native PowerShell collections are simple to create and easy to work with.   However, in large scale I find DataTable to perform better.   It requires more up front coding to get ready (columns, schema, populate rows) but then runs crazy fast.   Sample code below.

Enjoy! 

shades_smile

 

2207

 

PowerShell

People.CSV

© Copyright 2016
@ SPJeff

Return to Top ▲Return to Top ▲