• Powershell and writing files (how fast can you write to a file? )

    Hi there,

    I’ve been working for some time on a tool similar to PAL from mike lagase in order to automate the analysis of loadgen runs.

    While doing this I had to write large files with PowerShell and was not impressed with the result.

    I thought I’d share the problem and solution with the community.

    There are at least four ways of writing files with PSH:

    ·         use the ‘>>’' alias that calls into the out-file cmd-let

    ·         use export-csv

    ·         use .Net

    So let’s say I want to write 10000 lines into a file.

    Method 1: use the >> operator

    $s=”some text”

    1..10000 | % {
         $s >> ".\test.txt"
    }

     

    This way you actually open and close the file 1000 times.

    Method 2: use out-file to write an Array

    1..10000 | % {
        $a+=[Array] $s
    }

    $a | out-file ".\test.txt"

    This way actually writes once, using powershell.

    Method 3: use export-csv to write an Array

    $a=@()

    1..10000 | % {

          $a+=[Array] $s

    }

    $a | export-csv "t.txt"

    Export-csv can also write the array for you

     

    Method 4: use .Net StreamWriter to write the file

     

    $stream = [System.IO.StreamWriter] "t.txt"

    1..10000 | % {

          $stream.WriteLine($s)

    }

    $stream.close()

    The StreamWriter object from .Net also does the work nicely.

    Conclusion: how fast ?

    I tried all methods on my laptop, and here is what I got:

     Method

    Time to completion

    ‘>>’

    29 s

    Outfile and [Array]

    27 s

    export-csv

    22 s

    StreamWriter

    1.5 s

     

    Well results speak by themselves, if you are in a hurry, use .Net StreamWriter !

    Cheers,

    Next, check that with Powerhell v2!

    Guillaume

     

  • Where did my disk space go ? [very long directory names, system directories, symbolink and other hard links ...], meet xdir.exe

      [EDIT : posted a statically linked version to remove dependency over VCRT 9.0]

      Hi geeks,

    The first thing I do when I receive a new Microsoft-issued laptop is : buy the largest hard disk I can find to replace the one that shipped with it.

    But time goes ... and soon I wonder where all that disk space went.

    The other day one of my customers called me with that stance : "Guillaume, I've got a LUN on my SAN with 350GB I cannot account for".

    Where can there be content that is hidden from the system administrator himself ?

    1. System Volume Information is a folder with SYSTEM only  permissions (so administrators do not have access) that can contains many things from Single Instance storage data, recovery points, snapshots ...
    2. very long (> 256 chars) directories are not displayed by either explorer  or the command line because one needs to use the Unicode syntax when accessing those and for some reason our developpers did not.

    To make the story short my customer had VSS shadow storage defined on that LUN for snapshot of a different LUN, so using vssadmin delete shadows or vssadmin delete shadowstorage got his content back.

    But this incident convinced me to launch my could old C++ compiler to build a tool that deals with those kind of nightmares: meet XDIR.EXE

    XDIR is a combination of DIRUSE.EXE (the close parent of DU from Windows 2000 resource kit), RD (remove directory from the nt shell), and dir.

    BUT XDIR.EXE handles directories of any length, or permissions , it does try to follow symbolink links and you can use it to:

    • reclaim disk space : calculate the size of that particular directory and any subdirectory :
      try XDIR c:\
    • read a very long (or any) directory :
      try XDIR <your long directory> /DIR
    • delete a very long directory :
      try XDIR <your long directory> /RMDIR

    Options are :

    /MIN:xxx[GB|MB|KB|B] Minimum size to report (default is 500 MB)
    /LEV:nn Number of directory levels to display, default is 2, 0 will display all directories
    /FORCE known to the user of FILEACL, this will let you use your  SeBackupPrivilege and  SeRestorePrivilege to bypass the NTDS permissions (you need to be backup operator obviously)

    There are also a few other options that let you create an ANSI or UNICODE output file that came directly from FILEACL.

    Let’s try this: where is there content that weight more than 500MB on my C drive ?

    image

    What are those 2 GB in my System Volume Information ?

    image

    Well, that seems like a bunch of restore points to me. Wait, I’m running Windows Server 2008 and restore points are not a server feature ! Where are those kids from ? ohh, maybe from my old Windows XP dual boot !

    Let’s see

    image

    Files from last november? sounds like it’s the last time I booted my XP boot to flash my Windows Mobile, ok, let’s get rid of those (I’ll use the supported method of rebooting to XP and ask nicely that it delete those useless recovery points)

    let’s use another example:

    See that dir ?

    imageimage 

    Empty folder, big deal ?

    image

     

     

    why the heck, does xdir.exe think there is 1.167 GB in that dir ?

    image

    Well somebody twisted (me) has put 1 Gig into this stupidly long directory.

    Side Note : it is just impossible to go there with explorer so what can I do ? :

    • Create a JUNCTION point
      mklink  /J e:\temp\mount "e:\temp\01234567890123456789012345678901234567890123456789\01234567890123456789012345678901234567890123456789\01234567890123456789……
    • Create a DIRECTORY SYMBOLIC LINK
      mklink  /D e:\temp\mount "e:\temp\01234567890123456789012345678901234567890123456789\01234567890123456789012345678901234567890123456789\01234567890123456789……
    • use the good old SUBST from DOS :)

    if you just want to get rid of it use :

    XDIR <your long directory> /RMDIR

    like this:

    image

    add the /FORCE option if you do not have the rights on all folders (like with System Volume Information)

    Guillaume

  • Exchange Server 2007 rollups nightmares - automate the .config file modification

    Most of you Microsoft Exchange Server 2007 admins or Loadgen users as have been confronted with the problem described here : (Exchange 2007 managed code services do not start after you install an update rollup for Exchange 2007).

    Those who read the above technote probably felt as I did that every solution given where stupid. You know what? They are ! Who wants to configure Internet access on an Exchange server?  

    A very good description and the 'right" solution is given by Nino and the team on the EHLO blog : http://msexchangeteam.com/archive/2008/07/08/449159.aspx

    I've taken the liberty to automate the .config file modification for you, just run the script on your server and this will do the job:  

    ChangeExchangeConfigFile.PS1 will create, modify any .config file for :

    • Exchange services only (checking them against the service database)
    • Any exchange .exe file in the Exchange installation folder
    • or any path you will want to give

    Click here for the script itself.

    Edit : small bug correction on the script.
    Edit2: (version 1.2) let's home is is the last bug !

     

     

  • Hyper-V and Windows 2008 on my laptop - how to deal with wireless networking

    I received a few weeks ago my latest Microsoft laptop and as usual came the question of which OS to install it with.

    This time my focus was : go 64 bits and use our latest Windows Server OS, Windows Server 2008 which remarkably was getting out of the doors at that time.

    At the same time I had to prepare for my presentations at Techdays France 2008 which I decided to run on Hyper-V. The goal here was to have full portability between my home lab (also on W2K8x64 + HyperV) and my laptop, so if something went wrong with my laptop, I could hopefully connect to my home lab and go on with the demo.

    Hyper-V is great, the networking capabilities are getting better and better ... but as a Server Product it was not designed to use my WIFI network adapter, it is not possible to bridge a virtual network to a wireles network adapter. You can use Ben's solution (http://blogs.msdn.com/virtual_pc_guy/archive/2008/01/09/using-hyper-v-with-a-wireless-network-adapter.aspx),  but I do prefer my way, (otherwise I would not be posting) because it lets you keep control of your adressing.

    The goal is to create a Windows network bridge and include the virtual network in it.

    Here are my steps:
    1) Within Hyper-V Virtual Network Manager, create an "Internal Only" network, call it VMNET - it will also create a network adapter within Windows which device name will be the name of the Virtual network.


    2) go control panel->network connections Clic your wireless adapter, Ctrl-clic the VMNET bound adapter and then right-clic and choose "bridge"

    This way every virtual machines will be bridged to the wireless network and no NAT is involved, pretty handy for the next post ...

    Guillaume

  • How to use a smartcard within a Virtual Machine

    As you may know If you see Microsoft field people from time to time, we cannot stay unconnected from the Microsoft internal network (also known as CorpNet,  so Big Brother of us huh?) for long. Our VPN is protected with a SmartCard using only Microsoft technology of course. So my next step is to make my Virtual Machine "see" my smartcard reader.

    Neither Virtual PC, Virtual Server nor Hyper-V know how to redirect a USB port, si two solutions remain:

    • use the old, dusty serial smartcard my buddy Jean-Yves "mustache" Grasset once gave me and which I had to fight my wife for during the last "clean the house" event,
    • or remember that ... well ... Terminal Service does redirect the SmartCard itself to the TS !

    Since my "CorpNet VM" know has access to the Internet wherever I go I can enable "remote desktop" and TS in, use my smartcard and Voilà!