Search for and Replace Words in a CSV File by Using PowerShell

Search for and Replace Words in a CSV File by Using PowerShell

  • Comments 5
  • Likes

Summary: Learn how to search for and replace words in a CSV file by using Windows PowerShell.

 

Hey, Scripting Guy! QuestionHey, Scripting Guy! I have a comma-separated value (CSV) file that contains user names, user groups, and organizational unit (OU) assignments. The problem is that we are moving one of our office locations, and I need to quickly change the old location name to the new location name. I have been reading all of your articles this week, and I think I should be able to use Import-CSV to read the CSV file, and use a foreach loop to evaluate each row in the CSV file.

Inside there, I should be able to use an if statement to see if there is a match with the old name. If there is, I want to change it to the new name, and then export the CSV data to a new file. The problem is that this is not really working the way I want it to. For some reason, I appear to be picking up some kind of extra crap that is added by Windows PowerShell. I have spent all afternoon on something that should really be a simple one-liner. Help please. If I have to write a script to do this, I may as well go back to using VBScript. At least I have plenty of years experience using that language.

—GM

 

Hey, Scripting Guy! AnswerHello GM,

Microsoft Scripting Guy Ed Wilson here. One thing that is rather interesting about Windows PowerShell is that some things are really super easy. Other things appear to be difficult until you find the “correct” approach. One of the things I noticed in the 2011 Scripting Games is that some people work way too hard on their solutions. The other cool thing about Windows PowerShell is that in the end, if something runs without errors and something does what you need to be done, it is a solution. If I can write a bit of code in five minutes that is ugly but works, it is probably better than something that is elegant but takes me five hours to write. Don’t get too hung up on trying to craft the ultimate one-liner when you have a job to do, and that job does not entail crafting the ultimate one-liner.

On the other hand, there is something to be said for thinking about how to do things more effectively in Windows PowerShell. GM, I am imagining that your approach was to do something like this:

import-csv C:\fso\usersconsolidated.csv | foreach { If($_.ou -match "Atlanta") {$_.OU -replace "Atlanta","Cobb"}} | export-csv c:\myfolder\myfile.csv

This command does not work the way you want it to work … in fact, it does not really work very well at all because it does not produce the expected results. You are getting hung up with wrestling with the pipeline, getting confused with the Foreach-Object cmdlet (that is interrupting your pipeline), and messing about with the Export-CSV command that is not producing a CSV file at all.

While it is true that Windows PowerShell has some pretty cool cmdlets for working with CSV and for working with files, it is also true that at times, I want to be able to read the entire contents of a file into memory, and work on it at one time. The Get-Content cmdlet does not permit this; it basically creates an array of lines of text, which is great on most occasions.

To easily read the contents of a file all at once, I use the readalltext static method from the File class from the .NET Framework. The File class resides in the System.IO .NET Framework namespace. When using this class, I use square brackets around the class and namespace name. I can leave off the word system if I want to, or I can type it if I wish—it does not matter. If the word system is not present, Windows PowerShell will assume that the namespace contains the word system in it and will automatically use that when attempting to find the class. The .NET Framework namespaces are similar to the namespaces used in WMI because they are used to group related classes together for ease of reference. The difference is that the .NET Framework namespaces are a bit more granular. Therefore, if I am interested in working with files, directories, paths, file information, and other related items, I would go to the System.IO .NET Framework namespace and look around to see what is available.

After I find the File class, I can look it up on MSDN to see which methods it includes. The easiest ones to use are the static members (methods, properties, and events taken together become members) because using Windows PowerShell, all I need to do is to put the namespace/class name combination inside square brackets, use two colons, and the name of the method. And it works. Many times, the things a class provides are available somewhere else. For example, the File .NET Framework class provides a static method called exists. This method returns a Boolean value (true or false) that lets me know if a file exists or not. To use this method, I provide a string to the method. This technique is shown here:

PS C:\> [io.file]::exists("C:\fso\UserGroupNames.txt")

True

PS C:\> [io.file]::exists("C:\fso\missingfile.xxx")

False

I can accomplish the same thing by using the Test-Path cmdlet. This appears here.

PS C:\> Test-Path C:\fso\UserGroupNames.txt

True

PS C:\> Test-Path C:\fso\missingfile.xxx

False

It is always preferable to use a native Windows PowerShell cmdlet to do something, rather than resorting to .NET Framework, COM, WMI, ADSI, or some other technology—unless you have a compelling reason for doing otherwise.

A static method called readalltext is available from the file class, and it can be used in two ways. The first way is to supply a string that points to the path to the file to open. The second way is to specify the encoding of the file. Most of the time when reading a file, the encoding is not required because the method attempts to detect automatically the encoding of a file based on the presence of byte order marks. Encoding formats UTF-8 and UTF-32 (both big endian and little endian) can be detected. The result of using the readalltext method is that I get back a bunch of text in the form of a String class. The String class resides in the System .NET Framework namespace, and it contains a large number of methods. One of those methods is the replace method. I therefore add the replace method to the end of the readalltext method. The command to read all of the text in a file (CSV file) and to replace every instance of the word atlanta with the word cobb is shown here:

[io.file]::readalltext("C:\fso\usersconsolidated.csv").replace("atlanta","cobb")

The command to read a text file and replace words, and its associated output are shown in the following figure.

Image of command and associated output

To write this to a text file is simple. I can use the Out-File cmdlet as shown here:

[io.file]::readalltext("C:\fso\usersconsolidated.csv").replace("Atlanta","Cobb") | Out-File c:\fso\replacedAtlanta.csv -Encoding ascii –Force

The above is a single-line command that wraps in my Word document. I have not added any line continuation to the command. Keep in mind this technique is case sensitive. It will replace Atlanta, but not atlanta. The newly created text file is shown in the following figure.

Image of newly created text file 

GM, that is all there is to replacing values in a CSV file. Join me tomorrow for more exciting Windows PowerShell fun. TTFN.

I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.

Ed Wilson, Microsoft Scripting Guy

 

 

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • Hi Ed,

    this is a bit dangerous ... as I already commented to your last article ... but even MORE!

    First: The introductory lines:

    import-csv C:\fso\usersconsolidated.csv | foreach { If($_.ou -match "Atlanta") {$_.OU -replace "Atlanta","Cobb"}} | export-csv c:\myfolder\myfile.csv

    stated that the command does not produce the desired output, which of course is true! But a slight modification may work:

    import-csv C:\fso\usersconsolidated.csv | foreach { If($_.ou -match "Atlanta") {$_.OU = $_.OU -replace "Atlanta","Cobb"}; $_} | export-csv c:\myfolder\myfile.csv

    Your approach to read the whole text file does imply something unwanted: We loose the object structure that Import-csv would deliver! We end up in a stream of characters where we might exchange any occurance of "SearchMe" with "ReplaceMe" and that is ANYWHERE in the text!

    We don't have the OU property any more! And so we might replace the search string even in Lname, Fname or Group if it appears there! That's definitely unwanted ... so even if it isn't an "on first sight" solution, I would prefer to have objects and deal with the properties ... what you have been telling us all the time ... :-)

    Klaus.

  • @Klaus Schulte The prefered way to deal with CSV files is as objects! Windows PowerShell makes it so easy to do. In this particular example, I wanted to show another approach, and one that is "quick and dirty" ... it is a brute force way of doing things. Proper testing should always be done before doing anything. In addition, do not overwrite your source file, unless you have another copy :-) Thank you for pointing out that this may not be the best practice for all situations ...

  • @Klaus Schulte sorry hit post too quickly. I also like your fix to the opening scenario! It works, and is much more elegant than a brute force replace operation.

  • I totally agree with Klaus, your solution made me cringe while reading it...

  • I used this to remove ALL of the spaces in my -csv file. Works great!