Thoughts from the raised floor. Lewis Curtis Blog site

Working in an IT organization can be challenging.   There are exhausting late night emergencies and deadlines, unpredictable technical complexities as well as executive management commonly suffering from extreme ADD.

As this leads to countless hours away from family, friends and your own precious solitude,  you find those few who stand out willing to sacrifice everything to accomplish success.    "We're saved" you utter to yourself as disaster is averted once again or that cool project suddenly comes together. 

Periodically,   you parade the special person (or few) in front of staff for recognition and give them financial and psychological rewards.   Maybe, you even promote those few into leadership positions.    

As you sit in your office reflecting on the organization's accomplishment,   you joke; wishing just to have a few more of them.

As you are filled with confidence for the next year,  you have no idea:  Your IT organization is doomed. 

First, (and most obvious): You have created an organization dependent on these heros.

It is more difficult for heros to take vacation

It is often more difficult for heros to grow their career into other roles.

Heros are more likely to have strained family relationships

Non-heros often rely on heros to save the day.

Second, if Heros burn out and/or leave,  the IT organization is significantly crippled.

Third,  promoted heros often make poor managers (options)

Some have unreasonable expectations of their staff (they want them to be heros too)  or

They do all the work themselves

The end result: staff doesn't grow or become successful for the IT organization.

Fourth:  Eventually, heroic expectations eventually lead to catastrophic mistakes.   This is not only terrible for the organization,  but breaks moral and the spirit of the hard working employee you prize.

And you only have yourself to blame.   You created this culture, promoted these dependencies and burned out very good employees.     From every staff member (as well as their families) who has experienced the bad end of this strategic non-plan,  they hope you choke on the bonus you received this year.   (just kidding. :)  no - really,  I'm serious)

Simply: Heros ruin IT organizations.

It's a fact and IT leaders who promote this culture are to blame.

But there is a cure. 

Good Consultants will promote focusing on good operational processes and lifecycle management.   Establish a predictable, consistent system organized to handle complex IT solutions.   And they are right.   These are commonly called Operational Processes.  And they are one of the most important fields for an IT leader to understand.

However, there is a more simple way to understand this.    Look at your successes.  If it required a hero or heroic activity to become successful,  you screwed up.   It's your job as an IT leader to find these events and ensure processes and support for teams and empower them without breaking their backs.   

Also,  operational processes make a much bigger impact on systemic qualities than the actual technologies utilized in the solution.  Security, Scalability, Availability, Manageability, Reliability are usually more impacted by the quality of an organization's operational processes than the technical choices made. 

For example:  at Microsoft, we are building many of the world's largest datacenters.  Just at one datacenter, we deploy tens of thousands of servers a month.    Very few websites face the security threats and intensity that Microsoft manages everyday.   All while running some of the most energy efficient datacenter environments in the world.   And what is the secret ingredient: focused operational discipline.    Without it,  there would be no massive cloud capabilities today.  

I interviewed many teams at GFS (Global Foundations Services) under Michael Manos.    This is the group that owns all the datacenters in the world for Microsoft.  They have some of the highest retention rates in the company.    no-joke:  professionals at Microsoft continuously beg (no exaggeration)  to work for them.  Secret ingredient: good leadership that promotes world class operational discipline.  

Good IT Operational Processes:  the cure for Hero addiction...

Lewis

At TechEd Orlando,  I had an opportunity to interview Michael Manos (who is in charge of all the datacenters for Microsoft) about Leadership in the modern datacenter today.

Datacenter Leadership Interview.

Forrester recently published a report:

Infrastructure Architects Link Technology Strategy With Long-Term EA and Business Goals

It’s a decent article. 

For far too long many enterprise teams have viewed infrastructure architects as too low level not impacting or as close to the business.   That couldn’t be further from the truth.

I’ve always promoted that many times, infrastructure architects can make a  more substantial business impact on an organization than traditional  “enterprise” architects by a large margin (reducing cost and complexity in the organization,  reducing time to market, implementing significant technology and process reuse and standards).     of course I’m very biased…

In the past, most  Enterprise architect teams have only members with application and developer oriented skills supplementing with the traditional enterprise frameworks.       I find many leading Infrastructure architects today are traditionally older than their counterparts, many have a advanced degrees (even some with advanced business degrees) and demonstrates a deep enterprise impact perspective on the IT ecosystem.  

This article is signaling the start of a change in the enterprise architecture world.

I hope dialogue like this motivates more enterprise teams to adopt infrastructure leadership into their ranks in the future and see the value they have always brought.   

Lewis

I organized and participated on a Environmentally Sustainable Architecture panel last month.    There was some very interesting participants on the panel:

moderator: George Cerbone

Panelist: Michael Manos, Beth Humphreys, Kathy Malone, David Platt and myself.

This was the first panel to start including developers in the discussion with developers.   I invited David Platt to come in discuss.  He had some very interesting ideas and concerns to contribute.

Also, Kathy Malone is organizing the 1st ineta green users groups in Florida.   She has over 30 years experience on Environmental health and safety.     It was fascinating to hear her perspective.   I haven't followed the environmental health and safety perspective in this space as much.  I look forward to her future contributions into the ever expanding Green IT dialogue.

Beth Patton has been promoting environmentally sustainable datacenter best practices in the Chicago area for Microsoft.   This is where Microsoft is building one of their largest and most radically designed datacenters.

David Platt, author of the book "Why Software Sucks" was a great addition to the panel.   I knew he would be an excellent antagonist to really discuss why would this matter to developers.

of course Michael Manos is in charge of all the datacenters for Microsoft.   leading the way towards rebuilding Microsoft into a sustainable design.

video

watch and comment...

I spent a week vacation in Venice in April.   It was a beautiful place.   It was also the first time I went on vacation without my laptop (my wife's request).

Loosing the connection with the Web Collective...

This is more difficult than I thought.  I started going through withdrawal trying to enable my email to work on my GSM Windows Mobile phone via international roaming (it worked easily, although my wife was not pleased about it).   I just find it hard to disconnect.

Having passion about the projects I'm working on,  they don't stop for anyone who goes on vacation (and your responsibilities don't go away - work doesn't assign replacements for you when you are away).   I find it difficult to clear my mind of the things I'm working on or future endeavors.   My wife says I need to learn to meditate.    It took me half the week to start to clear my mind as we were enjoying our vacation.   after we returned, it took me a week to ramp back up (catching up on all the emails).

It hit me that maybe (because I think I'm not unique to this industry), maybe we need to rethink how we enjoy vacation with our families.   Since most of us can be engaged in our careers anytime, 24/7, we might have to create a personal program for ourselves to gear down and gear up so we can be in the moment of the vacation.   hmm. something to think about for next time.  Maybe the idea of week long vacation isn't reasonable anymore?  no, I don't want to believe this.  hmm

Venice.  the antipathy of creative destruction

It was interesting walking through Venice.   No Streets or Cars.    I thought the entire city was an giant museum that had decided not to develop past the 1500s.  Walk or Boat.  Thankfully, I didn't see any horses. :)

Creative Destruction promotes the concept of progress out of the destruction of the past (past thinking, design, architecture, etc...).  A great example is the beautiful city of Singapore (with strong laws making space for new architecture and new products).

Venice has survived and thrived on a theme that rejects creative destruction by embracing architectural concepts planted around the 1500s and earlier (there are a few exceptions of course).    They managed to utilize technical innovation (modern plumbing, electricity, telephones, Internet, cell phones, etc...) much as a very light condiment for a sandwich (and not be the sandwich itself).

I can't remember the number of churches we saw.   Venice is a city on the water with a lot of places to pray - if you are Catholic.   There was a historic church every 100 to 200 meters.  And my wife wanted to go into everyone of them.  Even my hotel, there was an historic church used as a storage area for the hotel that would be a national landmark in my country. 

We took every tour could get (often exhausting, reminding me how out of shape I am -which motivated me to join a gym).  A common theme of the tours- power, money and religion of course.  

We were surprised by the Jewish tour of seeing the first Jewish ghetto (the first use of the term as well) and the historic large population of Jews in Venice.  They were purposely quarantined into one area of the city (establishing the first ghetto).    Also, the tour discussed how the Germans took 200 to camps in WWII and only 8 returned.

On a lighter side, we ate well.   Some of the best food was inexpensive non-tourist pasta and pizza deep in the city well away from San Marco Square (main area of Venice where the tourists concentrate).

There was one place in San Marco which had excellent food.   the famous: Harry's Bar.   However, be warned.  The meal is very expensive.   But we had to go (my wife is a personal chef).   Harry's is famous for the invention of the Bellini and Carpaccio.

it was a nice vacation. :-)

I hope you had a good weekend. 

http://www.microsoft.com/environment/

As you know,  I've been heavily involved promoting architecture best practices for sustainability.   Microsoft has established a new environmental site which has a lot of promise.

Some highlights from the site for me:

• Windows Server 2008 Power Savings
• Datacenter best practices

However, this good site is only as good as the those passionate about the subject continues  to contribute best practices and insights.   Right now, most of these are from a little town in Redmond, Washington (I know for a fact that there are many very bright insightful people there).  

But it is my hope that many from around the world will begin contributing their experiences and perspectives to this site as this galvanizing theme continues to grow in this industry.

have fun,

Lewis

The demand for environmental impact information from organizations have increased significantly this last year.   and for good reason.   Customers, regulators, investors and partners are very concerned about carbon footprint, overall pollution and ecological impact.   This has put more companies under the microscope for their impact to environmental sustainability.

As a result, the Environmental / Greenness critic / analyst market is a lucrative field today.   And there is no shortage of critics on websites and TV stations.   However, as more self appointed greenness police enter the market,  it's important to investigate the narrow lens through which they judge organizations.   

I've noticed that an organization's commitment to environmental sustainability can be complex and more mature organizations usually use a multi-tier model.

If i were rating organizations impact on environmental sustainability, I would use a multi-tier model to investigate their work.

What are the number of impact areas you are targeting?

• Consumer experience
• Partner experience
• Worker productivity experience
• IT operational experience
• General Public experience

Then, you have to determine the metrics and progress you are going to make in focused impact areas.

Third, what processes, innovations and strategies are you leveraging to accomplish the goal.

Finally, you report your results.

However, most critics often use a very narrow lens to look at the "greenness" of an organization.  

Today, in the New York Times, the Climate Counts group gave an impressive rating to Google 55 while rating Microsoft at a 38.   They quoted Google's commitment to go carbon neutral.  

Google is a heavy user for energy and all of their green token projects have been tiny.  I predict they have spent more money marketing their green projects than the actual projects themselves.

Also, if they have a commitment to be carbon neutral, why don't they release their real carbon footprint numbers?  in the spirit of openness and "do no evil", why don't they disclose the real progress or allow the public to tour their centers to see the real work being done to improve environmental impact?

Apple was given a very low rating of 11.   I think it's comical in the interview with the New York Times, Apple blamed much of their carbon footprint on their users. 

So let talk about Microsoft:

Microsoft is one of the only massive web solutions companies that allows customers to tour their datacenters to see the real  environmental improvements to increase efficiency and decrease environmental impact.

From presentations from Microsoft's datacenter team to the public, it's explained how we measure and how granular we measure and what specific steps Microsoft takes.   I've worked for many large tech giants and at this point, I haven't seen a more open model to the public. 

also:

Microsoft as developed the most aggressive power saving features in the world for client and server computers.  There are significant power savings capabilities for consumers and administrators to control to reduce energy consumption of their operating system experience.

Microsoft's .Net platform has capabilities for developers to write power aware applications in WPF (windows presentation foundation) to reduce power drain on client systems.

In the last couple of years,(many would be surprised) Microsoft now offers some of the most consolidation infrastructure options to reduce the number of servers and clients in an IT organization.

Microsoft invests significant amounts of money into the Microsoft Research group to design solutions for consumers and corporations reduce environmental impact. 

Microsoft offers some of the most pervasive remote worker solutions in the world.

Microsoft has invested significantly in websites, concerts and public campaigns to  help consumers learn how to reduce environmental impact (much of it not relating to our product line).

In reality, it's easy to see how critics can pick apart organizations through their narrow lens.  I predict that we will see more of these models in the future.   But, I hope the environmental sustainability market matures to a better state than this.

Lewis

From conference presentations, customers meetings, political activity and internal debates,  I would like to clarify how the "green" Environmental Sustainability movement is different from the seventies and early eighties.

In the seventies and early eighties,  the green / environmentalist movement was focused on saving the earth from humans.  Popular publications like Gaia, Earth First, Greenpeace and the Worldwatch Institute consistently communicated negatively impacting the ecological stability of a region in light of industrial human activity.   

Environmental economists at the time countered with logical measurement environmental impact models as social impact externalities (what society was willing to pay for clean air, a wildlife habitat or stable ecosystem for endangered species).    They speculated on the level of pollution tolerated by a society or region (a magical social equilibrium).

In the last decade, many scientists as well as activists have promoted that while life is fragile,  the Earth is very resilient and will eventually get along just fine without us if we make too many mistakes along the way.

Beyond the temptation for purely positive public perception issue, today's environmental sustainability movement is universally supported by many extremely diverse political, social and economic interest groups.  And the reason.   it has nothing to do with saving "the Earth"

The modern environmental movement today is uniquely focused on one primary objective:  Save human civilization.

More specifically, this movement has demonstrated a passionate interest in saving geopolitical and economic stability for

• Established governments
• Global and regional markets
• Industries

If cities drown, becomes intoxicated with polluted air or farmlands and fishing lanes vanish , market places disappear and consumer and business wealth evaporates.   In the industrialized world, significant reductions in the ability to acquire basic living expectations (example: food, water, health care, communication and energy) by a large enough portion of the population quickly leads to predictable political and economic instability.    And that's the problem.

Many environmental focused scientists are predicting rapid dramatic climatic change which could significantly impact political and economic stability in the industrialized world.     And those same scientists agree that humans are the cause of this rapid acceleration.

As energy prices and environmental regulations are increasing,  organizations and governments are seeing the bottom line of of environmental damage beyond the magical social equilibrium.

Moreover, in the seventies and eighties,  the word "green" applied to environmental activism could be interpreted as a bad or good depending on one's political affliction.   Today, it's generally accepted as an important strategy for any government, industry or business.  It's not just for public perception,  but now for competitive survival.

Lewis

As the "Green" revolution is taking hold of the IT industry,  I'm noticing four core areas of execution in the market.  Most organizations focus on one (perhaps two areas) which showcases their strengths.  However, I discovered today, it takes paying attention to all four areas in your Environmentally Sustainability strategies.

Environmentally Sustainable

Optimization:

Reducing energy consumption and carbon footprint by optimizing the operating and development platform as well as the solution architecture.

example areas:

• Operating System energy consumption
• hardware (laptops, workstations, servers, mobile devices, etc..) energy consumption
• Datacenter Physical Facility energy consumption optimization
• Application Development Architectural optimization best practices to reduce resource consumption.

Consolidation:

Reducing energy consumption and carbon footprint by reducing the computing systems needed to accomplish the architecture effectively.

• Server / Client Hardware based vitalization
• Operating System vitalization
• Database consolidation
• Web Server consolidation
• Physical datacenter facility consolidation activities

Software + Services

Reduce energy consumption and carbon footprint by utilizing Software + Services capabilities for IT Systems as well as Human carbon footprint activities.

In the Datacenter world: another word: Transference: instead of designing a solution in-house, you leverage service environment to reduce your carbon footprint or energy consumption in your datacenter.

Online Presentation & Communication, Instant Messaging, Content management, Social Networks

Intelligence

Reducing energy consumption and environmental impact through Measurement, Forecasting, Communication and Management activity analyzing energy consumption and environmental impact with IT technologies

• Business Energy and Carbon Footprint metering and forecasting capabilities
• Business Energy and Carbon Footprint mgmt communication and charge back capabilities
• Environmental Business Intelligence activities (I refer it to as Sustainable Analytics)

in every project I've seen,  the focus areas usually relate to one or multiple areas from the above list.   Send me your thoughts and ideas on what you are seeing in the industry around environmentally sustainable solutions.

Lewis

From a chat with Dave O'hara today,  I thought I would blog some thoughts around datacenter energy consumption and some common confusion concerning costs.

Do organizations with dedicated datacenters save money when they install more efficient servers and reduce energy consumption? 

Short Answer:  Rarely

Why?

It all is associated with how most dedicated datacenters negotiate energy consumption with utility companies (operating cost issue).   Usually, they negotiate rates at blocks of energy consumption in fixed buckets.

Therefore rule #1:  do not run out of energy,  rule #2: do not leave energy supply stranded (pay for it and not use it).   In other words:  Overprovisioning and Underprovising

Overprovisioning:  you run out of energy in the datacenter (worst sin in the datacenter)

• TTM (time to market) is significantly impacted
• Datacenter systems become brittle  (small energy changes can down center)

Underprovisioning: you strand energy your organization already paid for: (very bad)

• company is wasting money (that's money that could be going into something useful)

There is fine balancing act to managing these issues in your datacenter.

At the end of the day,  we usually have a fixed bucket of energy consumption at a given rate which manage for our datacenter.

So why architect IT solutions which reduces the consumption of energy? 

Answer: Reducing the velocity of datacenter expansion in your organization (capital costs)

As your organization grows in scale and complexity at a given rate, the need build more and more competitive  IT solutions increases at a related velocity.  

As IT solutions expands, organizations need more datacenter capacity to accommodate the business's growth needs.   This expansion is only successful as long as the given IT solutions provide value above the capital of costs of building and operating new datacenter capacity. 

Some Challenges of building new datacenters:

• Regulations and oversight cost are increasing
• Cost of datacenter infrastructure are significantly increasing  (PDUs, Cooling solutions, etc..)
• Negotiated blocks of energy consumption are significantly increasing in price

Therefore, energy efficiency ultimately is about slowing the velocity of datacenter expansion for organization (capital costs).

If you can slow your build cycle from 1 new datacenter every 9 months to 1 datacenter to every 13 months can be a significant business value for your organization.    Slowing forecasted datacenter expansion velocity.

And ultimately, this is not only good for the environment, it's good for the bottom line. 

Lewis

In Feb 2008 edition of Harpers, Eric Janszen wrote an article titled "The Next Bubble" where he descibed the wave of investment in altnerative energy as the next bubble. http://www.harpers.org/archive/2008/02/0081908

While I aggree, there is good probability of bubble investment in alternative energy, this will hopefully drive down costs of alternative energy solutions.  But there is more to think about with this article.

I could see his point.   Yet, there are two economic bubbles happening.   One raising the value of the other.     Higher energy cost is the first bubble.      How big will this bubble be?    I’m not sure.  However, not one energy analyst has forecasted a downward trend in energy costs in the next 5 years.     We’ve seen this model before with past bubble activity;  Confident analysts predicting never ending higher speculative values.  

Because of the global nature of energy needs and political instability, the energy bubble might have a longer time cycle than most U.S. focused bubbles.     And this leads us to the second bubble: alternative energy.

It is true, that a possible alternative energy bubble is being uplifted by conspicuous consumption and public environmental reaction from higher profile climate change awareness campaigns.  However, the real muscle motivating many consumers and businesses into alternative energy planning is a  short term hyperinflation of the energy cost asset (or bubble : to use a term below).  

When gasoline prices soared after 1979, President Carter made moves to encourage alternative energy development and encouraged consumers to use less gasoline and electricity.    The popularity of alternative energy consumption became popular (from high MPG compact automobiles to outcries from the World Watch Institute to use bicycles) and many alternative energy businesses were formed as a result.   

But as the price of energy decreased significantly (the energy bubble), the 3 year alternative energy bubble quickly deteriorated.    SUVs and large homes became the symbols of conspicuous consumption.    Subscriptions to environmental magazines diminished.    Environmental groups were labeled extremists (although very few were) and their memberships dropped.  

Now, as a new energy bubble emerges,  environmental groups are regaining members,  the amount of environmental literature is increasing, and governments are pressing for more oversight, regulations and laws to manage energy consumption and encourage alternative energy use.

However, while the risk of this alternative energy bubble dropping  might be high when the ensuring energy bubble drops,  there is a difference between now and 1979.     The alternative energy bubble could be establishing long term regulatory, social and market externalities in the area of environmental accountability.     An example is disclosing carbon footprint and pollution impact of multi-national corporations.    

I predict that long after the price of energy drops and even an alternative energy bubble bursts, organizations will be still under an environmental regulatory microscope for the foreseeable future. 

Lewis

I was asked to prepare architectural best practices as we help others improve.  So from observing our own customers, partners as well as our own operations, here is a short initial list of 5 approaches.  Take a look and feel free to provide ideas and suggestions.

1) Understand where energy is consumed.& Use environmental monitoring to measure consumption and output and develop metrics

2) Use a holistic design approach to the architecture (carefully examines the environmental components in each tier, as well as the environmental impact on external systems supporting the overall solution) example: ultra-dense high utilization racks (from 8 kilowatts to potentially 12-22 kilowatts) significantly impacting the cooling architecture (and overall power consumption challenges)

3) Establish a focused set of system SKUs for each tier area to enforce energy efficiency and consumption standards, and environmental impact standards

example:

When purchasing hardware, ACPI 3.0 Systems can use advanced power management capabilities from Windows Vista and Windows Server 2008 to reduce energy consumption.   And for the server, consider reducing eliminating redundant power supplies for light stateless transactions and consider acquiring the most efficient power supplies available.

Environmental sustainability needs to be incorporated into the datacenter's change management and configuration management processes.

4)  Focus on the details of each tier of the infrastructure to reduce energy and environmental impact of key systems

5) Reduce architectural complexity

some examples:

Reduce the amount of tiers to reduce excessive system use

Aggregate tier systems through consolidation techniques.

6) Turn off systems when not needed.

Most understand the value of simply turning off a lite bulb when you leave the room.    The same analogy is true for IT operations.  A typical example is turning client machines off to powering down idle servers in the evening to save energy consumption.  

That’s it for now.   There are others.  But I'm interested in your your thoughts, observations and suggestions to reduce energy consumption and environmental impact with the IT solutions we deploy in our datacenters.

Lewis

Some Green IT predictions for 2008:

Prediction one:

2008 is the year that more realize Green IT is not a passing fad in the industry,  More will realize that Green IT is a permanent regulatory and operational reality in IT Architecture and Operations and it cannot be ignored.   Regulations and oversight as well as public scrutiny will increase in 2008 (as well as poor metrics in power consumption and carbon footprint).  We will see more laws and regulations, more audits, around the world.  

Because of this, many IT execs wanting one time quick fixes to get on the cover of their favorite industry magazine; will face tougher scrutiny from the public. 

Because this will be permanent reality for IT, more progressive organizations will understand the continuous commitment it takes to reduce power and carbon footprint and will hold decision makers and architects accountable for those metrics (as they do for scalability, availability, and security).     2008 will be a year of shaking out the green washing organizations and start to show some progressive organizations who are committed for the long run.

Prediction Two:

Companies who only rely on performance per watt (ppw) justifications for capital expenditures will see their power consumption increase (you read it right).

ppw has been a mainstay for vendors to justify new hardware and software it sells to IT organizations for the last thirty years.  

The logic goes like this:

"your (server/SAN/network/database/operating system) can do more work with the same amount of power,  therefore, you will need fewer of them,  hence you can reduce your power bill"

 Most Vendors are still parading the ppw marketing plan as their green answer today.   

So why doesn’t this argument work in the real world?    Answer: because it never factors in its impact on the velocity of demand as well as the impact of the environment which must now support it.

As technology capability increases, the velocity of people's demands of that technology will increase more.   Therefore the demand for more servers, storage and network capability will increase. This, in turn, will increase the demand for power.   This does not mention the cooling efficiency challenges of power dense racks (accounting for a substantial percentage of datacenter's power budget).

Question: Will better ppw metrics mean that the velocity of power demand will eventually decrease?  (Because things are more efficient)  Answer: no, it will actually increase.

This year, server consolidation through virtualization and blade systems will be more pervasive in 2008.   However, I predict that those who rely on this strategy alone (relying on the ppw model) will see their real power bills increase in the datacenters.

This might make an interesting pattern to investigate.

http://www.msnbc.msn.com/id/22266034/