Ken (Kenzaburo) Tamaru's Blog

What I was doing, RichCopy now..

Ken Tamaru — Thu, 21 Jan 2010 01:18:00 GMT

First of all I’d like to apologize for being away for so long and not actively updating this blog.

Today I will share what I was doing while I was away. My group is responsible for ensuring the quality of products aimed at enterprise customers as well as accelerating innovation through the engagement and joint activities with software and hardware partners. Over the past year I have been working almost exclusively on setting up our new facility, “Microsoft Otemachi Technology Center” in downtown Tokyo, Japan. This is a rather unique facility as it houses both MIC (Microsoft Innovation Center) and MTC (Microsoft Technology Center) as opposed to the usual model of having one facility dedicated to one center. MIC focuses on accelerating innovation and stimulating the local software economy with Microsoft technology, and MTC, which is for enterprise customers, “…is a collaborative environment that provides access to innovative technologies and world-class expertise, enabling you to envision, design, and deploy solutions to meet your exact needs” (MTC Web site). I was the project manager for every phase in setting up this new facility in Tokyo. My role not only involved driving the project, but I was also responsible to the entire design process from designing the office layout and infrastructure to setting up the switches, cables, etc. This new facility has 5 briefing rooms, 2 solution experience room, 1 envisioning room, 1 datacenter and 1 topology lab. Here is look like.

Datacenter

Briefing Room

Microsoft is committed to reducing carbon dioxide, and I spent a lot of time designing this new facility to minimize energy consumption, especially in the datacenter, which accommodates a total 40 server racks. In many cases, customers pay attention to server and storage energy consumption, but not the air conditioning infrastructure. At a typical datacenter, not container base, server and storage consumes less than 50% of total energy; the rest is consumed by lighting and air conditioning. Thus facility infrastructure is quite important for better PUE (Power Usage Effectiveness). Also this new facility hosts an environment for product testing, such as Exchange Server, SQL, and System Center’s management solutions. Network infrastructure was another key area.

Focuses are:

- Central management and monitoring of PDU (Power Distribution Unit) and Air Conditioners, not only server hardware and OS.

- Independent air conditioning control (per Server rack) to avoid unnecessarily cold lab.

- Maximum efficiency of cooling by minimizing the diffusion of hot air for high cooling efficiency. *Hot Aisle Containment

- Typical datacenter/lab configuration, not container, as our customers and partners come to visit and use them.

- Higher network bandwidth.

What we’ve chosen is APC’s datacenter solution. APC is Microsoft’s technology and solution partner, and they have rack and power solutions that satisfy the majority of the needs mentioned above.

Here is what the new center has deployed:

Summary of infrastructure

Rack

- APC’ Datacenter solution

Ø NetShelter Racks with HAC (Hot Aisle Containment)

Ø InRow cooling

Ø InfraStruXure PDUs

- HP ProCurve, Brocade and F5 for network solution.

Network Cables

- CAT6A

- OM3, ribbon cable for interconnection between racks and labs. (MPO/MPO)

Switches

- HP ProCurve, Brocade and F5 for network solution.

- 80G bps interconnection between labs.

- 10G network ready (CX4, SR)

- FCoE

Storage

- iSCSI (1000Base-SR 1000Base-T, 10G Base-SR [SFP+])

- Fibre Channel (2, 4, 8G bps)

- FCoE (10G bps)

HAC (Hot Aisle Containment) to minimize the diffusion of hot air for high cooling efficiency.

RichCopy

Long path, reparse point, some missing command options, etc will be addressed in next release. It will be released in a couple of weeks.

RichCopy 4.0.216をリリースしました

Ken Tamaru — Sat, 13 Jun 2009 08:52:00 GMT

マイクロソフト社内で非常に多くの人に使用頂いているRichCopyなのですが、社外でも様々に取り上げられ少し驚いています。マイクロソフトはグローバルにビジネスを展開していますが、同様に社内のインフラもグローバルに展開されています。その最も重要な一つにネットワークがあるのですが、データ転送はいつも悩まされるところです。「遅延の大きいWANでいかに効率よくファイルを転送するか」、これがこのツールを作成する動機となっています。最近RichCopyを取り上げていただいたメディアは以下の通りです。

@IT [アットマーク・アイティ（IT Proの方であれば良くご存知のインターネットメディア）
窓の杜　（オンラインソフトを紹介するインターネットサイト）
週間アスキー

ファイルコピーは単純な作業であるが故に、そこに求められるものは大きいのだと感じました。
ところで、バージョン4.0.216をリリースしました。このリリースでは以下のバグを修正しています。

1. /CSA /CSD /CSG /CSO /CSS の使用、オプションダイアログの設定内容に関わらず、セキュリティ情報がコピーされない。
2. コマンドラインから実行した場合、一部のログ情報が正しく記録されない。
3. コマンドラインから実行した場合、まれにコピー先のパスが化ける。
4. ファイルフィルタリング機能に関する記述がヘルプファイルに無い。

更新されたRichCopyをインストールする際は、必ず既存のバージョンをアンインストールしてください。新しいバージョンは、前のバージョンと同じ、以下よりダウンロードする事が出来ます。

http://download.microsoft.com/download/f/d/0/fd05def7-68a1-4f71-8546-25c359cc0842/HoffmanUtilitySpotlight2009_04.exe

RichCopy Build 4.0.216 has been posted to the Microsoft Download Center.

Ken Tamaru — Thu, 11 Jun 2009 07:57:00 GMT

I am so surprised there are so many IT professionals who are looking for rich featured file copy tools. Since RichCopy became publicly available from the Microsoft download center, I am receiving tons of emails such as questions, bug reports, feature requests. I am sorry that I am not able to respond to all of your e-mails, however I will answer most of them in my blog. Some of you may have noticed that RichCopy package on Microsoft download center has been replaced with a new build which is Version 4.0.216. There is no additional features, but the following bugs have been fixed in this release.

1. Cannot copy security information when /CSA /CSD /CSG /CSO /CSS options are specified or equivalent options are set in the option dialog.
2. Logging information is truncated when RichCopy is executed from the command line.
3. Destination path got corrupted when RichCopy is executed from the command line. (rarely happens..)
4. There is no description of file and directory filtering command parameters in the helpfile…

Currently installed RichCopy has to be removed before installing the new version.
http://download.microsoft.com/download/f/d/0/fd05def7-68a1-4f71-8546-25c359cc0842/HoffmanUtilitySpotlight2009_04.exe

One thing I found is,
There must be many users who use RichCopy to copy only updated files. Most of users assign only 1 thread for directory search; however you can dramatically accelerate the performance of source and destination comparison by assigning multiple threads, especially this works well when files are distributed to multiple directories as RichCopy assign 1 thread to each directory search, not a tree.

Here is an example. (local to local, but different storage)

(1 million files in source and destination)
Thread # for Directory Search
1 about 10 minutes
2 about 6 minutes
4 about 2 minutes
8 about 1 minutes

32/64-bit detection

Ken Tamaru — Mon, 04 May 2009 09:24:00 GMT

The Codebase of RichCopy version 4.0 is completely different from version 3.x. RichCopy 3.x consists of COM components, and the UI is a just an interface for COM components. It supports automation and shell scripting; however component registration is required, and it is not light enough on Vista because of UAC. RichCopy version 4.0 is designed to run standalone and actually without updating registries or installing multiple components. If you copy RichCopy.exe or RichCopy64.exe into your working directory, and then it should run.
32/64-bit detection: RichCopy.exe determines whether the platform is 32-bit or 64-bit. On a 64-bit platform and RichCopy64.exe is present, and then RichCopy64.exe is executed instead. Here is actual code RichCopy uses.

hKernel32  = GetModuleHandle(TEXT("kernel32.dll"));
if (hKernel32)
{
  pIsWow64Process   = (LPISWOW64PROCESS)GetProcAddress(hKernel32, (LPCSTR)"IsWow64Process");
  pGetNativeSystemInfo = (LPGETNATIVESYSTEMINFO)GetProcAddress(hKernel32, (LPCSTR)"GetNativeSystemInfo");
  if (pIsWow64Process)
   (*pIsWow64Process)(GetCurrentProcess(), &fWow64);
  if (pGetNativeSystemInfo)
   (*pGetNativeSystemInfo)(&si);   FreeModule(hKernel32);
}
else
{
  GetSystemInfo(&si);
} .....
  if ((sizeof(LONG_PTR) == sizeof(DWORD)) &&
  (si.wProcessorArchitecture == PROCESSOR_ARCHITECTURE_AMD64))
{

Multi-threaded file copy, RichCopy

Ken Tamaru — Thu, 30 Apr 2009 08:47:00 GMT

Keith Combs convinced me to start blog (http://blogs.technet.com/keithcombs/). What for? I often receive e-mails asking about RichCopy, but it changed when Joshua Hoffman wrote an article about RichCopy in TechNet Magazine, April, 2009 issue. (http://technet.microsoft.com/en-us/magazine/2009.04.utilityspotlight.aspx) Keith covered this in his article, and consequently got tons of question on his blog (http://blogs.technet.com/keithcombs/archive/2009/03/22/richcopy-bulk-file-copy-tool-released-get-it-here.aspx). If I understand correctly, he screamed for help “I am not a support contact of RichCopy”, and asked me to start bloging on the TechNet.

What is RichCopy? Please visit Keith’s blog and 2009.4 issue of TechNet magazine. RichCopy was originally developed more than 10 years ago when I was working on Exchange Server 4.0 (a.k.a Touchdown project) and some other projects as a development lead. One of painful job was to copying latest build to my test machines over slow and large latency network. 10M bps was still majority of edge switches and NetBEUI as protocol at that time. RichCopy was developed to address those issues, especially to put another layer of packet transaction window. In my blogs, I will cover best practice of RichCopy configuration, QA, and I will also write about implementation for example how RichCopy supports over 64 threads while WaitForMultipleObjectEx() does not support more than 64 threads (MAXIMUM_WAIT_OBJECTS).