Thoughts from the raised floor. Lewis Curtis Blog site

The demand for environmental impact information from organizations have increased significantly this last year.   and for good reason.   Customers, regulators, investors and partners are very concerned about carbon footprint, overall pollution and ecological impact.   This has put more companies under the microscope for their impact to environmental sustainability.

As a result, the Environmental / Greenness critic / analyst market is a lucrative field today.   And there is no shortage of critics on websites and TV stations.   However, as more self appointed greenness police enter the market,  it's important to investigate the narrow lens through which they judge organizations.   

I've noticed that an organization's commitment to environmental sustainability can be complex and more mature organizations usually use a multi-tier model.

If i were rating organizations impact on environmental sustainability, I would use a multi-tier model to investigate their work.

What are the number of impact areas you are targeting?

• Consumer experience
• Partner experience
• Worker productivity experience
• IT operational experience
• General Public experience

Then, you have to determine the metrics and progress you are going to make in focused impact areas.

Third, what processes, innovations and strategies are you leveraging to accomplish the goal.

Finally, you report your results.

However, most critics often use a very narrow lens to look at the "greenness" of an organization.  

Today, in the New York Times, the Climate Counts group gave an impressive rating to Google 55 while rating Microsoft at a 38.   They quoted Google's commitment to go carbon neutral.  

Google is a heavy user for energy and all of their green token projects have been tiny.  I predict they have spent more money marketing their green projects than the actual projects themselves.

Also, if they have a commitment to be carbon neutral, why don't they release their real carbon footprint numbers?  in the spirit of openness and "do no evil", why don't they disclose the real progress or allow the public to tour their centers to see the real work being done to improve environmental impact?

Apple was given a very low rating of 11.   I think it's comical in the interview with the New York Times, Apple blamed much of their carbon footprint on their users. 

So let talk about Microsoft:

Microsoft is one of the only massive web solutions companies that allows customers to tour their datacenters to see the real  environmental improvements to increase efficiency and decrease environmental impact.

From presentations from Microsoft's datacenter team to the public, it's explained how we measure and how granular we measure and what specific steps Microsoft takes.   I've worked for many large tech giants and at this point, I haven't seen a more open model to the public. 

also:

Microsoft as developed the most aggressive power saving features in the world for client and server computers.  There are significant power savings capabilities for consumers and administrators to control to reduce energy consumption of their operating system experience.

Microsoft's .Net platform has capabilities for developers to write power aware applications in WPF (windows presentation foundation) to reduce power drain on client systems.

In the last couple of years,(many would be surprised) Microsoft now offers some of the most consolidation infrastructure options to reduce the number of servers and clients in an IT organization.

Microsoft invests significant amounts of money into the Microsoft Research group to design solutions for consumers and corporations reduce environmental impact. 

Microsoft offers some of the most pervasive remote worker solutions in the world.

Microsoft has invested significantly in websites, concerts and public campaigns to  help consumers learn how to reduce environmental impact (much of it not relating to our product line).

In reality, it's easy to see how critics can pick apart organizations through their narrow lens.  I predict that we will see more of these models in the future.   But, I hope the environmental sustainability market matures to a better state than this.

Lewis

From conference presentations, customers meetings, political activity and internal debates,  I would like to clarify how the "green" Environmental Sustainability movement is different from the seventies and early eighties.

In the seventies and early eighties,  the green / environmentalist movement was focused on saving the earth from humans.  Popular publications like Gaia, Earth First, Greenpeace and the Worldwatch Institute consistently communicated negatively impacting the ecological stability of a region in light of industrial human activity.   

Environmental economists at the time countered with logical measurement environmental impact models as social impact externalities (what society was willing to pay for clean air, a wildlife habitat or stable ecosystem for endangered species).    They speculated on the level of pollution tolerated by a society or region (a magical social equilibrium).

In the last decade, many scientists as well as activists have promoted that while life is fragile,  the Earth is very resilient and will eventually get along just fine without us if we make too many mistakes along the way.

Beyond the temptation for purely positive public perception issue, today's environmental sustainability movement is universally supported by many extremely diverse political, social and economic interest groups.  And the reason.   it has nothing to do with saving "the Earth"

The modern environmental movement today is uniquely focused on one primary objective:  Save human civilization.

More specifically, this movement has demonstrated a passionate interest in saving geopolitical and economic stability for

• Established governments
• Global and regional markets
• Industries

If cities drown, becomes intoxicated with polluted air or farmlands and fishing lanes vanish , market places disappear and consumer and business wealth evaporates.   In the industrialized world, significant reductions in the ability to acquire basic living expectations (example: food, water, health care, communication and energy) by a large enough portion of the population quickly leads to predictable political and economic instability.    And that's the problem.

Many environmental focused scientists are predicting rapid dramatic climatic change which could significantly impact political and economic stability in the industrialized world.     And those same scientists agree that humans are the cause of this rapid acceleration.

As energy prices and environmental regulations are increasing,  organizations and governments are seeing the bottom line of of environmental damage beyond the magical social equilibrium.

Moreover, in the seventies and eighties,  the word "green" applied to environmental activism could be interpreted as a bad or good depending on one's political affliction.   Today, it's generally accepted as an important strategy for any government, industry or business.  It's not just for public perception,  but now for competitive survival.

Lewis

As the "Green" revolution is taking hold of the IT industry,  I'm noticing four core areas of execution in the market.  Most organizations focus on one (perhaps two areas) which showcases their strengths.  However, I discovered today, it takes paying attention to all four areas in your Environmentally Sustainability strategies.

Environmentally Sustainable

Optimization:

Reducing energy consumption and carbon footprint by optimizing the operating and development platform as well as the solution architecture.

example areas:

• Operating System energy consumption
• hardware (laptops, workstations, servers, mobile devices, etc..) energy consumption
• Datacenter Physical Facility energy consumption optimization
• Application Development Architectural optimization best practices to reduce resource consumption.

Consolidation:

Reducing energy consumption and carbon footprint by reducing the computing systems needed to accomplish the architecture effectively.

• Server / Client Hardware based vitalization
• Operating System vitalization
• Database consolidation
• Web Server consolidation
• Physical datacenter facility consolidation activities

Software + Services

Reduce energy consumption and carbon footprint by utilizing Software + Services capabilities for IT Systems as well as Human carbon footprint activities.

In the Datacenter world: another word: Transference: instead of designing a solution in-house, you leverage service environment to reduce your carbon footprint or energy consumption in your datacenter.

Online Presentation & Communication, Instant Messaging, Content management, Social Networks

Intelligence

Reducing energy consumption and environmental impact through Measurement, Forecasting, Communication and Management activity analyzing energy consumption and environmental impact with IT technologies

• Business Energy and Carbon Footprint metering and forecasting capabilities
• Business Energy and Carbon Footprint mgmt communication and charge back capabilities
• Environmental Business Intelligence activities (I refer it to as Sustainable Analytics)

in every project I've seen,  the focus areas usually relate to one or multiple areas from the above list.   Send me your thoughts and ideas on what you are seeing in the industry around environmentally sustainable solutions.

Lewis

From a chat with Dave O'hara today,  I thought I would blog some thoughts around datacenter energy consumption and some common confusion concerning costs.

Do organizations with dedicated datacenters save money when they install more efficient servers and reduce energy consumption? 

Short Answer:  Rarely

Why?

It all is associated with how most dedicated datacenters negotiate energy consumption with utility companies (operating cost issue).   Usually, they negotiate rates at blocks of energy consumption in fixed buckets.

Therefore rule #1:  do not run out of energy,  rule #2: do not leave energy supply stranded (pay for it and not use it).   In other words:  Overprovisioning and Underprovising

Overprovisioning:  you run out of energy in the datacenter (worst sin in the datacenter)

• TTM (time to market) is significantly impacted
• Datacenter systems become brittle  (small energy changes can down center)

Underprovisioning: you strand energy your organization already paid for: (very bad)

• company is wasting money (that's money that could be going into something useful)

There is fine balancing act to managing these issues in your datacenter.

At the end of the day,  we usually have a fixed bucket of energy consumption at a given rate which manage for our datacenter.

So why architect IT solutions which reduces the consumption of energy? 

Answer: Reducing the velocity of datacenter expansion in your organization (capital costs)

As your organization grows in scale and complexity at a given rate, the need build more and more competitive  IT solutions increases at a related velocity.  

As IT solutions expands, organizations need more datacenter capacity to accommodate the business's growth needs.   This expansion is only successful as long as the given IT solutions provide value above the capital of costs of building and operating new datacenter capacity. 

Some Challenges of building new datacenters:

• Regulations and oversight cost are increasing
• Cost of datacenter infrastructure are significantly increasing  (PDUs, Cooling solutions, etc..)
• Negotiated blocks of energy consumption are significantly increasing in price

Therefore, energy efficiency ultimately is about slowing the velocity of datacenter expansion for organization (capital costs).

If you can slow your build cycle from 1 new datacenter every 9 months to 1 datacenter to every 13 months can be a significant business value for your organization.    Slowing forecasted datacenter expansion velocity.

And ultimately, this is not only good for the environment, it's good for the bottom line. 

Lewis

In Feb 2008 edition of Harpers, Eric Janszen wrote an article titled "The Next Bubble" where he descibed the wave of investment in altnerative energy as the next bubble. http://www.harpers.org/archive/2008/02/0081908

While I aggree, there is good probability of bubble investment in alternative energy, this will hopefully drive down costs of alternative energy solutions.  But there is more to think about with this article.

I could see his point.   Yet, there are two economic bubbles happening.   One raising the value of the other.     Higher energy cost is the first bubble.      How big will this bubble be?    I’m not sure.  However, not one energy analyst has forecasted a downward trend in energy costs in the next 5 years.     We’ve seen this model before with past bubble activity;  Confident analysts predicting never ending higher speculative values.  

Because of the global nature of energy needs and political instability, the energy bubble might have a longer time cycle than most U.S. focused bubbles.     And this leads us to the second bubble: alternative energy.

It is true, that a possible alternative energy bubble is being uplifted by conspicuous consumption and public environmental reaction from higher profile climate change awareness campaigns.  However, the real muscle motivating many consumers and businesses into alternative energy planning is a  short term hyperinflation of the energy cost asset (or bubble : to use a term below).  

When gasoline prices soared after 1979, President Carter made moves to encourage alternative energy development and encouraged consumers to use less gasoline and electricity.    The popularity of alternative energy consumption became popular (from high MPG compact automobiles to outcries from the World Watch Institute to use bicycles) and many alternative energy businesses were formed as a result.   

But as the price of energy decreased significantly (the energy bubble), the 3 year alternative energy bubble quickly deteriorated.    SUVs and large homes became the symbols of conspicuous consumption.    Subscriptions to environmental magazines diminished.    Environmental groups were labeled extremists (although very few were) and their memberships dropped.  

Now, as a new energy bubble emerges,  environmental groups are regaining members,  the amount of environmental literature is increasing, and governments are pressing for more oversight, regulations and laws to manage energy consumption and encourage alternative energy use.

However, while the risk of this alternative energy bubble dropping  might be high when the ensuring energy bubble drops,  there is a difference between now and 1979.     The alternative energy bubble could be establishing long term regulatory, social and market externalities in the area of environmental accountability.     An example is disclosing carbon footprint and pollution impact of multi-national corporations.    

I predict that long after the price of energy drops and even an alternative energy bubble bursts, organizations will be still under an environmental regulatory microscope for the foreseeable future. 

Lewis

I was asked to prepare architectural best practices as we help others improve.  So from observing our own customers, partners as well as our own operations, here is a short initial list of 5 approaches.  Take a look and feel free to provide ideas and suggestions.

1) Understand where energy is consumed.& Use environmental monitoring to measure consumption and output and develop metrics

2) Use a holistic design approach to the architecture (carefully examines the environmental components in each tier, as well as the environmental impact on external systems supporting the overall solution) example: ultra-dense high utilization racks (from 8 kilowatts to potentially 12-22 kilowatts) significantly impacting the cooling architecture (and overall power consumption challenges)

3) Establish a focused set of system SKUs for each tier area to enforce energy efficiency and consumption standards, and environmental impact standards

example:

When purchasing hardware, ACPI 3.0 Systems can use advanced power management capabilities from Windows Vista and Windows Server 2008 to reduce energy consumption.   And for the server, consider reducing eliminating redundant power supplies for light stateless transactions and consider acquiring the most efficient power supplies available.

Environmental sustainability needs to be incorporated into the datacenter's change management and configuration management processes.

4)  Focus on the details of each tier of the infrastructure to reduce energy and environmental impact of key systems

5) Reduce architectural complexity

some examples:

Reduce the amount of tiers to reduce excessive system use

Aggregate tier systems through consolidation techniques.

6) Turn off systems when not needed.

Most understand the value of simply turning off a lite bulb when you leave the room.    The same analogy is true for IT operations.  A typical example is turning client machines off to powering down idle servers in the evening to save energy consumption.  

That’s it for now.   There are others.  But I'm interested in your your thoughts, observations and suggestions to reduce energy consumption and environmental impact with the IT solutions we deploy in our datacenters.

Lewis

Some Green IT predictions for 2008:

Prediction one:

2008 is the year that more realize Green IT is not a passing fad in the industry,  More will realize that Green IT is a permanent regulatory and operational reality in IT Architecture and Operations and it cannot be ignored.   Regulations and oversight as well as public scrutiny will increase in 2008 (as well as poor metrics in power consumption and carbon footprint).  We will see more laws and regulations, more audits, around the world.  

Because of this, many IT execs wanting one time quick fixes to get on the cover of their favorite industry magazine; will face tougher scrutiny from the public. 

Because this will be permanent reality for IT, more progressive organizations will understand the continuous commitment it takes to reduce power and carbon footprint and will hold decision makers and architects accountable for those metrics (as they do for scalability, availability, and security).     2008 will be a year of shaking out the green washing organizations and start to show some progressive organizations who are committed for the long run.

Prediction Two:

Companies who only rely on performance per watt (ppw) justifications for capital expenditures will see their power consumption increase (you read it right).

ppw has been a mainstay for vendors to justify new hardware and software it sells to IT organizations for the last thirty years.  

The logic goes like this:

"your (server/SAN/network/database/operating system) can do more work with the same amount of power,  therefore, you will need fewer of them,  hence you can reduce your power bill"

 Most Vendors are still parading the ppw marketing plan as their green answer today.   

So why doesn’t this argument work in the real world?    Answer: because it never factors in its impact on the velocity of demand as well as the impact of the environment which must now support it.

As technology capability increases, the velocity of people's demands of that technology will increase more.   Therefore the demand for more servers, storage and network capability will increase. This, in turn, will increase the demand for power.   This does not mention the cooling efficiency challenges of power dense racks (accounting for a substantial percentage of datacenter's power budget).

Question: Will better ppw metrics mean that the velocity of power demand will eventually decrease?  (Because things are more efficient)  Answer: no, it will actually increase.

This year, server consolidation through virtualization and blade systems will be more pervasive in 2008.   However, I predict that those who rely on this strategy alone (relying on the ppw model) will see their real power bills increase in the datacenters.

This might make an interesting pattern to investigate.

http://www.msnbc.msn.com/id/22266034/

Corporate IT initiatives to reduce environmental impact and power consumption is here for the long run. Executives are allocating time, energy and money to invest in Green initiatives. Governments are allocating research, regulations and suggesting laws toward Green Datacenter efficiency. Consumers, policy makers and industry influentials are promoting Green Datacenter models.

We didn't see laws and regulations promoted for SOA, or Agile design, or Web 2.0 or SaaS, … Ten years from now, those initiatives might not even exist, but commitments to reduce environmental impact and power consumption will continue to be important for organizations.

Many view the Green Datacenter as a product feature checklist to gain their one time win. But that is an unfortunate illusion. While new technology from the industry will help, it does not replace the ongoing architectural and process commitment needed.

Green Datacenters = An Architectural Commitment, not a product Strategy

For example, A virtualization or a blade environment product decision has the potential to reduce power consumption. But if there are no processes or architectural guidance to go with it, it can encourage server sprawl and eventually increase power consumption. And of course, increasing rack power density without a aligned cooling architecture is a recipe for datacenter disaster.

Environmental Impact and Power Consumption is becoming a crucial architectural systemic quality metric:

In the past, IT architects gave too little attention to security: Eventually suffering the consequences. Environmental Impact and Power are quickly becoming pervasive architectural issues with new initiatives.

Traditional IT Architecture Goals (motivations for the profession) Encourage IT reuse, Reduce IT Complexity, Align Stakeholders, Optimize functional and non-functional (systemic quality goals) and spend the organization's money wisely.

Architectural decision points

Reducing power consumption is obvious. Gone are the days of measuring datacenters by square foot of space. Now, datacenters are increasingly sized by mega watt. More efficient technologies with new capabilities are being promoted as the silver bullet. But energy savings is a much more complex architectural issue from architectural power management capacity planning techniques to optimizing operational processes and facilities design.

Reducing environmental impact is more challenging. But this industry initiative isn't called the power reduction initiative. It's called the Green initiative for an important reason. There is a consensus that serious negative environmental repercussions are the consequence of man made pollution. From the atmosphere to the soils and oceans, governments, partners, consumers and industry organizations want companies to have a more positive impact on the environment.

The most common environmental impact measurement is carbon footprint (usually measured in tons: from energy source and amount to manufacture and logistics operations) . How are you architecting the solution to reduce the organization's carbon impact on the atmosphere. For Architecture design: fewer systems which are more energy efficient, reducing the degree of functional decomposition in the design (doing more work with less code and systems), leveraging services from carbon neutral environments. There are many examples out there and this a growing field that will be part of the IT Architect vocabulary.

The world is changing for IT Architects and we must develop new skills. The days of architecting a system without power and environmental considerations are numbered. Just like other Architectural skills, Green metrics and vocabulary will become pervasive and we will be accountable for this important IT Architectural issue in our organizations for the long run. It's already happening.

For the next couple of months, Working with a MS alum: Dave O'Hara, I'm working towards a Microsoft comprehensive Green Datacenter Strategy.   It is wonderful to see so many fellow employees very supportive in this effort.  We're scheduled to give an internal presention to inteternal employees at Techready 5 (internal conf at the end of July).   This presentation will focused on having a energy consumption strategy in the datacenter, what the industry is doing, what customers expect from us and ideas for Microsoft for the future.

Lewis

For a while, developmental and operational environments have rarely worked well together.   Developers design something and then throw it over the wall for the infrastructure team to design an environment to support it.   Then it’s thrown over the wall for operational teams to deploy and monitor it for the rest of its life.

What’s the wall?   It’s usually email, conference calls and planning meetings.   Most of the tools between these groups work well with each other.  Furthermore, even the languages utilized between teams are alien between each other (with significant hours spent on translation activities between teams).   Within a regular silo based solution, groups treat each other’s domains (storage, server environments, components of the applications, data center services, solution design, etc…) as “do not enter” zones of mutually acceptable areas of exclusion.    These areas of comfortable ignorance lay the foundation for the house of cards which awaits.

Therefore, like sheep diseased with industry groupthink, so many thought we could force everyone to agree on the same management APIs, protocols or same programming language to be the magic which cures the quagmire of datacenter complexity.   The problem was these approaches were focused on commonality within a group (example, CIM for datacenter management, WS-I for developers).  Given the big promise, complexity should have decreased.   However, most would agree datacenter complexity is instead, growing.

And with service oriented architecture, the complexity of the datacenter is only increasing.   Services must depend on other services developed and managed with different technologies and with different levels of maturity.   Furthermore, the quality of these designs and management models between silos can be radically different.  Like a deeply troubled psychiatric patient, datacenters are becoming more brittle with pressure to manage co-dependent services which managers and architects often do not control.  

We live in illusions of control in the datacenter.   There are versions of technologies and designs we inherit from others.  We rarely control the politics of product selection or alignment from the solutions we are asked to provide infrastructure for.   We rarely influence the technological evolution of the products and technologies we use today.   Even senior management has less control than most think about operational budgets, personnel and skill levels supported in a datacenter.    Unfortunately for some, happily promoting this illusion of control in blissful ignorance lays the ground work for the worst.

The truth is much of our blissful ignorance is not a technical design problem, but a human problem.   Instead of agreeing to use the same versions of the same products, people use the tools and products which work best for them.    Fighting this natural tendency is a losing battle.   The question is: do we embrace this assumption and design solutions to adapt or fight the losing battle of command and control to promote the illusion?

Microsoft made a decision several years ago to embrace diversity for developers.  This is why Microsoft decided to invest in .net in a radically different way from others.   At the time, the Java community assumed most wanted to change operating systems by promoting developers write to the same programming language specification and use a JVM (Java bytecode was used for the Java language. In my 5 years at Sun, I’ve never seen any other languages utilized in the mainstream environment attached to Java bytecode). This is not to slam Java.  It was just addressing a very different business problem (the platform).    

Instead, Microsoft focused on the human problem.  Microsoft found that people rarely port code from platform to platform in their companies (even between JVMs).  However, developers often utilize different programming languages to solve different business challenges.   With examples today like dynamic languages, this trend will only increase.   Humans are adaptive and utilize languages and structures which work for them at that specific time.  In other words, Microsoft realized that promoting a single programming language was an illusion of control.   The key for .net was to enable programmers of different languages to be successful with Microsoft’s platform (which they could control).

Yet, there is still a problem for infrastructure architecture and solution architecture.  They still don’t really communicate well.   There have been positive standards based trends in the datacenter.   WS-Management will help tremendously with interoperability between management environments and diverse application and operating systems (not to mention past work with CIM and others).  Microsoft adopted WS-Management with W2K3R2 and future management products.   Yet, this was never designed to address the “throw it over the wall” infrastructure design and management problem today.

Now Microsoft is promoting the architectural specifications for the Systems Modeling Language (SML).   SML is different.   It’s one of the first proposed standards focused on the relationships between system components and even between application services.   SML does not assume people design in the same language, leverage the same operating platform or network or use the same management or deployment environment.  It assumes controlling those areas are an illusion.  SML is focused on mapping architectural relationships between these diverse systems up and down (hardware to application) and across (between services/applications). 

1)      It will allow developers to list the dependencies between a designed application and the application server configuration (example: IIS configuration for .net applications), network dependencies, database dependencies, hardware dependencies and even external service dependencies.

2)      It will allow infrastructure architecture teams to map constraints (datacenter standards) to drive reuse and reduce complexity and costs.  Furthermore, being able to map out constraints in the SML model will drive security standards (example constraint: “no databases can be in the DMZ”).   Developer tools designed to absorb SML constraints can establish and enforce this at design time to reduce defects before they reach test and production.

3)      Solutions mapped utilizing the SML will be able to be more accurately tested, deployed and optimized in the datacenter.   Furthermore, with relationship dependencies more accurately mapped, datacenter teams will be able to reduce their MTTR (mean time to repair) when troubleshooting an SML modeled solution with a more accurate health model.

For example, what happens when outbound email stops working in a company?   The operational team checks the outbound MTAs (Mail transfer Agents) for problems.   If the outbound MTAs are ok but are just not processing any mail, then the team might realize the probable bottleneck could be outbound DNS.   Once outbound DNS is repaired, mail operates normally again.  However, the budget and team which managed the email design often did not have anything to do with the budget or team involved with designing the DNS servers (and vice versa).   Yet, there is a clear relationship which should be recognized and mapped.  Currently, little exists in the world of standards to do this.

While I know it’s popular for the press to be skeptical, Microsoft recognizes the significant diversity of technologies in our datacenter.   You can see this in most of Microsoft’s products now aligning much better to open standards and structures.  This blog’s example:  with the support of partners, competitors and strangers in the industry, Microsoft announced it will promote SML as a common standard to help tear down these walls of ignorance between architecture teams and allow us to live with a little more control in this world of constantly evolving technologies in the datacenter.   Besides building this model into Visual Studio, Microsoft is committing this model in most of its next generation management products.   Furthermore, if you want to see how past Novell or Sun interoperability announcements will be more realized in the future, just look to the power of SML.  This is part of the vision of a new Microsoft and I believe it’s a pretty good idea.

You can find out more information at

http://www.microsoft.com/windowsserversystem/dsi/serviceml.mspx

http://www.microsoft.com/windowsserversystem/dsi/default.mspx

Microsoft renaming SDM to SML

http://www.microsoft.com/presspass/press/2006/jul06/07-31SMLPR.mspx

There are tremendous opportunities and cool new services coming from Web 2.0 like services today which our user base is becoming more excited about.  However, as we all know, we must understand the impact of these services on the integrity of the information technology ecosystem.

One such interesting service is Swarm. www.swarmthe.com 

I recommend caution with Swarm.   It collects anonymous (from the subscriber) browsing activity.    This can include your browsing activity (web addresses and IP addresses) of internal corporate sites.   This might be useful for hackers.  This macro spyware is used innocently today as a cool neat way to see aggregated activity on the web.   Wait till hackers find ways to use it for passive intelligence gathering on a targeted vector (say: your corporate internal layout).  Furthermore, hijacking another person’s session via swarm doesn’t seem to be addressed.

Also, Currently, there is no explicit documentation on Swarm.  Hmm.

In a recent security issue, the swarm team disabled gmail links because of the problem of signing on with someone else’s account.

(which would have been CNN news if it involved Microsoft hotmail)

At the end of the day,  it's always buyer beware (even if it's free).

Some have asked about what are good approaches for examining the complexity (or trying to figure out how to associate operational staffing numbers for a specific datacenter) tough question.

Here is what I used in the past to decompose this question (some stupid approaches and some which worked better….)

Pure work related activity:

1. Budget for a higher amount of operational staff and train, automate and optimize for a smaller amount.   (it’s never that easy and it’s cop out approach) dumb and not recommended

2. Some take past CMMI automation, process, structure audits from past 3rd party consultants to justify head count. Lower the CMMI score,  the higher the FTE count is the common approach.   (again,  never that easy and often a newbee approach that gets you into trouble later)  not recommended

3. The stuff/technology per person measurement: example: Patch/Monitoring by physical processor / box is ridiculous with today’s complexities:   Why?  Because the modern datacenter is more complicated:

Some Examples of those complexities:

Virtual servers managed, virtual SANs managed, virtual networks managed, 2rd party management systems, SLAs and OLAs (Operational Level Agreements), Silos managed, Business Applications managed,  Velocity of New apps introduced, Old apps retired, Complexity of current regulations/laws,  Glue Complexity between data center services, Security systems for authentication, authorization, confidentiality, privacy, etc.. , Risk tolerance levels, physical center constraints (including HVAC),  discipline, capabilities of operational staff as well as discipline, capabilities and knowledge base of development staff.   Of course,  politics, organizational dynamics, fiscal budget models: capex/opex relationships etc…

Thinking about how everything impacts the OLAs

OLAs (Operational Level Agreements)  often, the systemic qualities for specific business applications

Generally, in most ISP datacenters,  every extra 9 in the availability measurement can increase operational FTE significantly. From past work with organizations needing extremely high HA with triple redundant everything (examples: HBA cards, clusters, load balancers, trunking network cards, etc…) for HA even during maintenance periods can promote significant Operational staff requirements.   This strategy promoted (by sheer statistics) significant hardware component failures but kept continuously high HA (redundancy promotes an inverse relationship between component reliability and solution availability: which equates to more resources needed for higher availability rates).   While the specific architecture was continuous,  the amount of human work was nothing less than extraordinary.  This was further compounded with the requirement for mandatory access control needs.

Auditing all of these areas isn’t easy.

If these approaches didn’t work all that well,  what has been working?

Measuring Operational Complexity

Better way:  decompose how many specific elements would be measured for MTTR (mean time to repair) (often in the OLA structure).   MTTR elements can be anything that a MTTR metric must address (software, hardware, etc…)   The higher the element count utilized for MTTR (from hardware, systems, virtualized stuff, etc…),  the higher the complexity,  the more work operational staff  needed to responsibly manage it.   I’ve found this a fairly accurate complexity measuring approach transcending the technologies and trends of the day.

Measuring Operational Scale:

(not the same as the Systemic Quality: Scalability)

For each solution, application, etc…  count the diversity and number of external entities (some use the term actors from UML use case diagrams but this is not always as inclusive) utilizing the solution

Then.

Calculate the growth rate (or shrink rate) of the diversity and number of external entities utilizing each solution in the datacenter.

Then.

Calculate impact rate on common data center services from the growth rate of the solutions.

(example: DNS, NAT, Routers, E-Mail, Middleware, Backup and Recovery Systems, Disaster Recovery Operations, etc…)

It’s simple,  Greater degrees of scale and scope traditionally increase operational staff requirements.

The trick is accurately forecasting this with the customer based on lots of stated assumptions (hint)

The Marketing Stuff: Automation solving World Peace of Datacenter automation promise:

Today,  many companies, including us, are promoting significant automated capabilities to reduce operational staffing needs as scale and complexity are increased (example: “we can manage more servers per person than they can sales pitch so you should build an SOA datacenter with us.” …)    It’s true that some automated technologies in isolation have helped datacenter management for all operating system vendors.   Research with the largest and most advanced datacenters in the world have not yet demonstrated the realization of fewer headcount or work while delivering higher availability rates promise (the opposite is often measured).  And of course, servers per headcount is a meaningless metric without understanding the real context with which it was built (as most who read this blog already know. J )   We will have to digest SLA strategies into highly automated and extremely focused OLA  MTTR measurable and manageable elements.   Like the perfect product promise, a magical process alone will also not solve this problem.   It takes the best architecture with the best products with the best processes and good people in the right organization (if all the stars in the galaxy align argument).   Or at least a real commitment to align those stars (which is what I recommend when really taking on an automation strategy)  And this is one of the many reasons why this promise is so often not realized.   Today, the best approaches automate to reduce cost and complexity impacting a highly focused subset of MTTR elements.   The sad humor in the datacenter: while a couple of MTTR elements are often automated to levels requiring less human activity,  the sheer complexity of the automation system many times requires a pile of new MTTR elements no else thought of (of course )  An interesting way to measure the value of datacenter automation tools and techniques: the number of MTTR elements automated to the number of MTTR elements introduced.

This leads to another approach: looking for datacenter architecture stability = Velocity of entering and exiting MTTR elements per quarter.   A high number can often indicate chaos (no matter how cool the stuff they are buying).

Just some of the approaches I’ve used in the past.

Hope this helps…

Lewis