Follow Us on Twitter
by MichaelF on August 04, 2006 05:50pm
UML (User Mode Linux) and Forensic Analysis
(Special thanks to Dan Simonton for the testing and writing in support of this tech tip)
Processes running under UML will have no access to the hosting system, accept where explicitly allowed. Because of this UML is an ideal candidate for operating a honeypot. While processes running in UML have no outside access to the host operating system memory or filesystem; hypothetically, if an attacker managed to break out of userspace into some section of the host filesystem, they could do further damage on the host . Best-practice demands that host access be limited within the UML instance wherever possible.
For any of this to be practical, obviously some services would need to be established. We’ll just assume these are already in place and forwarding iptables rules setup on the host. For instance, to forward inbound http connections to your UML instance:
iptables -t nat -A PREROUTING -i eth0 -p tcp -s 0/0 \ --dport 80 -j DNAT --to-destination (uml ip):80
This process could be repeated for any other service you wish to run. Just be sure the appropriate destination port is specified both with the “--dport” option and also at the end of the command. Essentially you are instructing all inbound traffic to port 80 to be forwarded to your UML. The same process could be repeated for ssh,ftp and others.
Typically, the first order of business for an intruder with root access is to wipe out log files. It is best therefore to have log files written to the host (or another remote) machine. To achieve this, the host system’s syslogd daemon must be configured to receive inbound logs. Add “-r” to the runlevel script of the host machine where it invokes syslogd the correct path is: /etc/rc.d/syslog. On the client machine, add the following to /etc/syslog.conf:
*.* @yourhostmachine (note: @ipaddress will work also)
Now inbound connections to the UML honeypot and activity on the honeypot can be investigated through log files on the host machine.
Conveniently for the purpose of running UML as a honeypot, tty_logging of UML into a directory on the host machine is built as an option into the kernel. The simplest way to achieve this is to add the following to the kernel command line:
tty_log_dir=dir
This way, even if they zap the shell log files on the UML, you will still have an account of their activity on the machine.
There are a few quick and common methods of checking running activity. The following two should be familiar to anyone with a relatively basic understanding Linux, but we’ll mention them here for propriety’s sake:
ps auxwww (check running process table)
netstat –lvnap|less (check open sockets, associated process, and user ids)
In the past, whenever I’ve found any strange binaries (sometimes named something really vague or obscure), I’ll run the following command, sift through, and evaluate the output:
strings (filename)|less
An all-to-commonly overlooked tool for inspecting a system is “lsof” (list of open files). It can be used to check file-to-file access, files listening on a socket and evaluate the state of a running process. It is helpful to know the normal running health of a system for comparison when using lsof. For a quick check of a specific process:
lsof –p (pid)
To get socket info on a process:
lsof -i -nP|grep -i (process name)
To protect against potential outbound denial-of-service attacks, it might be prudent to explicitly declare hosts you wish to allow outbound ICMP traffic to (the host ip being one for example) and deny everything else. This can be done on the host by adding the following rules to iptables. You can add as many “ACCEPT” rules as you need, just be sure to put them before the “DROP” rule.
iptables -A INPUT -p icmp –s (uml ip) –d (host ip) –j ACCEPT iptables –A INPUT –p icmp –s (uml ip) –d (other ip) –j ACCEPT iptables –A INPUT –p icmp –s (uml ip) –d 0/0 –j DROP
Similiarly, you could block potential outbound syn-flooding:
iptables -N syn_flood iptables -A INPUT -p tcp --syn –s (uml ip) -d 0/0 -j syn_flood iptables -A syn_flood -m limit --limit 1/s --limit-burst 3 -j RETURN iptables -A syn_flood -j DROP
A quick search of http://sourceforge.net or http://freshmeat.net will quickly realize a vast sea of various analysis tools. Provided the disk image size for your is adequate, any of these can be copied to the drive image or simply downloaded once you have the UML instance running. A few useful tools are:
Tripwire: Useful for monitoring data integrity. In a nutshell, it takes a snapshot of your system binaries (or other specified directory), creates a checksum, runs routine system integrity checks against it, and reports any deviation.
The Coroner's Toolkit: A suite of utilities for checking running process and file/filesystem information, recent changes and other such information.
Snort: Snort is so prevalent, it almost needs no description. Still, it is one of the best tools for traffic analysis and intrusion detection. To accurately provide a description that does this monster justice would be a blog unto itself. There is a great FAQ on their website:
http://www.snort.org/docs/faq/1Q05/
Chkrootkit: A utility for identifying rootkits installed on the system.
This is but a small (microscopic) primer into a much larger world of intrusion detection and integrity analysis, but we hope some may find this useful. We will likely delve into these subjects in the future.
by jcannon on July 06, 2006 03:06pm
Another guest blog this week from Identity Management Program Manager, Shamit Patel: ---------------------------------------------
Hi, Last week, we released two new utilities to help customers achieve UNIX / Windows Interop. The first is a set of utilities and the SDK for the Subsystem for UNIX Architecture (SUA) in Vista Beta 2 & Longhorn. For those unaware, SUA is a native subsystem residing on top of the Windows kernel, just like the Win32 subsystem. It provides the basic infrastructure to run UNIX-based applications and scripts on Windows Vista (Ultimate and Enterprise) and Longhorn Server.
We've also released the UNIX-side components for Identity Management with UNIX. This essentially provides the utilities which enable password sync between Windows and UNIX environments. These are the UNIX-based utilities to enable successful synchronization.
I realize many of you may not be testing Vista or Longhorn, but for those who are, or have corporate testing, we would love to hear your feedback on the product, scripts and documentation.
Thanks all, Shamit
by jcannon on July 03, 2006 02:30pm
I see my last couple posts were about ambiguity, so I thought today I’d blog about something, IMO, that is not ambiguous at all—and the topic would be a fitting hat tip to Sara and Korby and all the folks involved with CodePlex. Brief background: We had to buy our own combination padlocks on our lockers in my high school. I used to forget the combination all the time (—I still have nightmares about that). I finally solved this by writing my combination in hex on the back of the lock. (I figured there was only one other kid in my class who would know what 0F was in base-10, so if anything was ever missing, I’d know where to look. ) I tell this little anecdote because it made me think about the lack of a community of folks with similar interests in my little world back then. The only reason I knew hex* went back years earlier to a similar lack of community: I couldn’t get a game I was writing on my Commodore 64 to do some things fast enough in BASIC, so I asked my Dad what else I could do and he explained what Assembly language was, and from then on there were lots of nights when I was supposed to be asleep, sitting there in my pajamas, banging away in 6502 Assembly land—by myself. This was long before the concept of a home modem would have ever occurred to us, never mind the modern Internet’s enablement of community and collaborative development--but I can’t help but wonder what a difference it might have made to me (never mind the quality of that game!) if there had been a more readily accessible community of folks interesting in collaborating and mentoring at that time. What does this have to do with praising open source developers? This week, inspired by CodePlex, I was looking back at two of the most important studies of the motivations of open source developers. In the two studies (Ghosh in 2002 and Lakhani (PDF) in 2004—both are available online), although slightly different sets of questions were asked, by a notable margin the leading responses were “Learn and develop new skills” and “Share knowledge and skills” (Ghosh) and “Code for project is intellectually stimulating to write” and ‘Improve programming skills” (Lakhani). What’s even more striking about this is comparing these types of motivations—about learning and sharing—with more “confrontational” motivations. Developers could choose multiple answers in both studies, and, for example, in the Lakhani study four times as many volunteer developers chose “Improve programming skills” as a reason for joining an open source community than “Dislike proprietary software and want to defeat them.” To be clear, anybody’s reason is valid to them--but I am a person who would rather learn than win. That’s true when I write code, it’s true when I play soccer; I think that is a good way to view the world—and from all the research I’ve seen, the evidence is compelling that folks who voluntarily participate in open source development communities place very high value on learning and sharing their knowledge with others. I don’t have comparable data at hand, but I’m willing to believe it is well higher than the average person in the population at large. And for that—kudos. I think that means there are far more opportunities for kids like I once was not just because of technological advances, but because of people—maybe people like you reading this post. *I actually can’t remember if I stumbled across hex first in Traveller, where, as I recall the descriptive strings for character attributes and planets where in hex—come on, don’t snicker, you know you played it too…
by admin on May 02, 2006 12:08pm
Swimming is my cure for jet lag. I am currently at the MES 2006 (Microsoft Executive Summit) in Mumbai (Bombay), India – an annual event for the top 250 CIO’s in India. I’ve been here a couple days and have been waking up at 3am, so my cure has been a pre-dawn swim in the hotel pool. The hotel I’m staying at has a nice pool, right next to Powai lake and it’s protected forest, where I later learn is a natural home to leopards and alligators, which keeps me alert while swimming solo.
MES 2006 is a great opportunity to meet many of the top CIOs, IT decision makers and partners in India and I’ve been enjoying it immensely. I did a presentation on our platform strategy, people-ready businesses, and in particular how we think about ‘coopetition’ in this strategy. One of my favorite things about this job is talking to customers about their IT environments, issues and dreams. What has been fascinating about my conversations here in Mumbai has been the many different ways customers have designed and architected for interoperability. From banks to manufacturing companies to consulting services, almost every customer or partner I’ve met with has an interesting story about interoperability in their IT environment. No surprise really, heterogeneity is part of any large IT system, but the recurring theme I’ve noticed here is the pragmatism and clarity of focus on where and when interoperability is needed. And if you’ve ever spent time on the roads in Mumbai, you’ll realize that interoperability is part of everyday life!
One Microsoft partner I had dinner with explained a very large, multi-tier system they build and sell which uses Windows, Unix systems, and a mainframe – all for one application (it is a large and critical application, so this isn’t really overkill for what they do). Although they want to eventually migrate from the mainframe for cost reasons, they have chosen technologies to get the job done as best suited their needs and skills. And – importantly – they factor interoperability into every architectural plan, RFP, or design that they think about – it’s as important to them as feature functionality or testing. It’s a core part of their maturity model. So what do they look for to qualify something as ‘interoperable’? Open and mature standards that have industry wide acceptance. They also understand the difference between open standards and open source, and gave me a very lucid walkthrough of the differences. Simply put, they explained that open source is a development and distribution model and open standards are specifications that can be applied to interfaces and technologies to enable data exchange. It is that clarity that, I believe, has helped them to design for interoperability with their eyes wide open.
One more thing. I had a customer meeting where I heard a great description of IT value. We were talking about software utilization, the dreaded ‘application backlog’ that many CIO’s face (CIO magazine has a great column on this here). When I asked about their deployment experiences with Microsoft software, the customer told me a story about their instant messaging deployment. Within 48 hours of deploying Microsoft Live Communications Server for instant messaging and collaboration, they had over 16,000 people utilizing the product. He then said, “Listen Bill, it is actually quite simple, when I can deploy software that immediately 16,000 people start using on their own because it’s important and useful to them, that is value.” Clear and simple definition of value: people use it on their own volition - something we should all remember.
Although swimming does help jet lag, coffee is equally important, so it’s now time to go find some. Until next time, may you avoid stray leopards. -Bill
by jcannon on July 05, 2006 03:17pm
Free open source management projects have existed for years, as illustrated by nagios and webmin, and exist as BYOC (bring your own console) free alternatives to commercial management systems from HP, BMC, CA, IBM and Microsoft. In the last few years, we've seen a rise in commercial software companies moving to support Linux and heterogeneous environments - including but not limited to Centrify, Vintela (Quest) and Centeris, three vendors with whom we've worked in the lab.
It makes good economic sense to make money managing a free product - after all, Microeconomics 101 will tell you that commoditizing your complements maximizes revenue. Sell a database? Then make the operating system and application server free. IBM's move into open source can be seen in this perspective (free operating systems on for-profit hardware and services) as can HP's (with management software revenues thrown into the mix). The same logic should apply to management, especially given the relative lack of enterprise-class open source management software. While nagios is impressive, the fact that it has been used to manage 5,000 node systems alone does not make it enterprise-class.
Recently the Open Management Consortium was founded to unite free/libre open source management projects around a common vision for what management systems should be capable of, and under a common philosophy of open source software. Founders include Qlusters, EmuSoftware, Zenoss, and Ayamon. They also have a list of OSS management projects. Notably, they don't mention OpenSSI as a cluster management technology.
Open Source can be taken to apply to management in several ways:
Each of these layers is open to displacement by open source software, some more easily than others. Agents and adapters seem to me to be the best fit for the typical open source development model - where it's easier to serve the long tail of different endpoints than under standard commercial rules. Consoles and monitors, while at the most basic levels of logging, parsing, alerting, and displaying are well-understood, are areas of deep research and increasingly rarified technology. The developments in the area of event aggregation and scalable management UIs require significant directed investment (and Matt Asay has disagreed with me on this before) in which commercial software companies have an advantage.
A few Port 25 readers have contacted me about building open source integrations between Microsoft products and OSS management technology - as well as OSS projects and Microsoft management technology. For both of these categories, it makes good sense to me and I'd like to see them developed at www.codeplex.com, where we've built an infrastructure for the community to build open source projects.
In the management arena, where we spend significant time in the lab testing different approaches, I'd be happy to spend money and time helping to test or develop projects on Codeplex. Drop me a note if you have something cooking and would like some help or direction.
Cheers, Sam
by MichaelF on January 02, 2007 05:52pm
Just a quick note to let folks know that we are going film the Festivus Roundtable (if you're confused as to what I'm talking about click here) next week so if you have questions, please take a moment to submit them before Monday January 8, 2007.
You can leave comments here, on the original post, or via email.
Based on submissions to date we expect to have an interesting discussion and if you enjoy this one we'll wash, rinse and repeat (but promise not to wait for the next imaginary holiday).
-michael
by admin on May 05, 2006 03:08pm
Over the past four weeks, we've been very excited by the activity on Port 25 - the participation has been very encouraging and has largely kept a positive & healthy perspective. Conspiracy theories aside, this is exactly the kind of thoughtful conversation we want to continue to have. And our key message will not change - it is a heterogenous world in computing and customers of all sizes will always expect solutions to "just work." Interoperability between platforms - such as Windows, Linux and UNIX, is key to meeting that expectation.
But getting IT to "just work" is not easy - by any standard. Most of us live this challenge every day, understanding full well that today's solutions will only be dwarfed by tomorrow's challenges.
That's why we're interested in hearing about your biggest - and smallest - technical challenges. Send them in - your toughest pains, trivial pet peeves - and we'll try & answer them. We've already started here and here. But we know there are a ton more! So starting today, we're calling all technical questions.....send e-mail directly to our lab.
We'll read through each & every one - and start building out responses to help all of us "just work" better together :)
-Kishi
by admin on June 05, 2006 03:51pm
Infrastructure Management and Strategic Design Part 1: Domains Portfolio Management
Thank you all and especially everyone who has been giving us very productive feedback on what you'd be interested in seeing us write about. Your input helps us tune our content to what’s important to YOU, our viewers. On that note I am kicking off a series of blogs towards Infrastructure Management encompassing suggestions, situations and pointers towards managing and maintaining a large-scale heterogeneous infrastructure. I’d like to start today’s blog on an “unsung” hero of the Infrastructure Service but yet one of the most important– Domains Portfolio Management.
Why is this important: Well, think about this for a minute - EVERYTIME someone types a url in their browser, it translates to someone having secured that url at some point and working with a registrar to purchase and lock it. I can literally write a book on how painful it has been for so many companies to sometimes purchase, acquire or secure something as small as a url and YET, the entire web presence of a company depends on registration of a simple url, a central place which everyone goes to.
Domains Portfolio Management as I understand it, encompasses managing all gTLD’s (Global top level domains such as .com, .net etc) and ccTLD’s (Country Code Top Level Domains such as .au, .in etc.) across the board. This involves managing, securing and expanding your Domain Portfolio credentials to give us a business advantage and to understand and leverage this often overlooked strategy. Let’s say you’re working for an organization that is geographically dispersed in major regions around the world. If you want to maintain, develop and build a strong Internet Presence portfolio, the dynamics of that space are changing very rapidly with the introduction of several new top level domains e.g. .mobi etc. I have spent a significant amount of time in my past life overseeing the several aspects of Domain Management and here’s the top three suggestions I would like to share with you:
1. Manage your Domain Portfolio: If you have more than 100 domains in your Domains Portfolio, it is worth every penny to build a database around these domains and enter essential information like expiration date, registrar details etc in it. This in itself will add years to your life as a Domain Portfolio Administrator. Other helpful suggestions are choosing more than one registrar for your portfolio. The phrase “don’t put all your eggs in one basket” has never been more relevant than at this point. Why you ask, well, think in terms of ‘TDRP – or Technical Disaster Recovery Planning”. You want to put measures in place such that you always have a backup Registrar/Registry that you have been working with and knows your portfolio intimately 2. Secure your Domain Portfolio: Sometimes, we “over-prepare” and I have seen Domain Portfolio Managers registering every single url in every single country in the world, even if they’re not known and/or are not selling a product or service in that region. So what does a good Domain Strategy look like. Well, start by maximizing your gTLD assets, make sure you have fully locked and secured your Domain Assets. Work w/ your Registrar and have them do a comprehensive Domain Audit and review for you. You are more than likely to get a visualization of where most of your Domain investment lies. Sit down w/ your Business and Marketing Team after that to review the findings and align your Domain presence with your core business goals. e.g. if your org is heavily centric in a specific continent or country, make sure your domain assets are maximized for that region. 3. Build a strategy around your portfolio: Once you have a solid strategy around what areas, regions and assets you need to invest in, you are more than likely to get your share of squatters (those who buy popular domain names and then sell them to the highest bidder), type-squatters (those who make their money by buying domains resembling popular / heavy-hitter names ) and some clear violations of your company trademark name. While you’re combing through these issues, “PICK YOUR BATTLES”. If a specific ccTLD is not that important to your company’s business expansion plans, just let it go. But make sure you invest heavily in keeping and securing the Domain Assets that your business model takes for granted.
1. Manage your Domain Portfolio: If you have more than 100 domains in your Domains Portfolio, it is worth every penny to build a database around these domains and enter essential information like expiration date, registrar details etc in it. This in itself will add years to your life as a Domain Portfolio Administrator. Other helpful suggestions are choosing more than one registrar for your portfolio. The phrase “don’t put all your eggs in one basket” has never been more relevant than at this point. Why you ask, well, think in terms of ‘TDRP – or Technical Disaster Recovery Planning”. You want to put measures in place such that you always have a backup Registrar/Registry that you have been working with and knows your portfolio intimately
2. Secure your Domain Portfolio: Sometimes, we “over-prepare” and I have seen Domain Portfolio Managers registering every single url in every single country in the world, even if they’re not known and/or are not selling a product or service in that region. So what does a good Domain Strategy look like. Well, start by maximizing your gTLD assets, make sure you have fully locked and secured your Domain Assets. Work w/ your Registrar and have them do a comprehensive Domain Audit and review for you. You are more than likely to get a visualization of where most of your Domain investment lies. Sit down w/ your Business and Marketing Team after that to review the findings and align your Domain presence with your core business goals. e.g. if your org is heavily centric in a specific continent or country, make sure your domain assets are maximized for that region.
3. Build a strategy around your portfolio: Once you have a solid strategy around what areas, regions and assets you need to invest in, you are more than likely to get your share of squatters (those who buy popular domain names and then sell them to the highest bidder), type-squatters (those who make their money by buying domains resembling popular / heavy-hitter names ) and some clear violations of your company trademark name. While you’re combing through these issues, “PICK YOUR BATTLES”. If a specific ccTLD is not that important to your company’s business expansion plans, just let it go. But make sure you invest heavily in keeping and securing the Domain Assets that your business model takes for granted.
In conclusion, I would like to say that there’s much more riding on your domain portfolio than most would ever imagine. Building an asset database, building a close professional relationship with your Registrar and always staying on top of the Domains namespace are hard suggestions I would like to make after having lived through managing this service. Have a great week ahead and THANK YOU for tuning in!!