Using NMAPI to Access TCP Payload

Using NMAPI to Access TCP Payload

  • Comments 5
  • Likes

The TCP Payload often carries data that you want to access directly using the Network Monitor API. Below I will detail how to do this using a simple C++ example and the NMAPI.

Why Not add a TCP.Payload Field?

The TCP Payload can carry all types of payloads depending on the protocol that rides on top of TCP. Most often these represent other protocols, but you might not care about the protocol and instead want to see the payload size or payload data directly. You might think that you could access TCP.Payload to access this data, as this is a valid data field. However, TCP.Payload is only instantiated when no other protocol consumes the data. And in most cases, our parsers are complete enough to attempt to parse the data further. This is a limitation of how NPL works, and means we need to find another way to get the payload data.

Why Not use Property.TCPPayload?

Now there is a property, see this blog for more info on properties, called Property.TCPPayload that you could potentially use. The limitation is that it only works with ASCII or UNICODE data. So for binary information the data does not read properly into the property.

The Solution

The solution is to find the TCP payload depending on the TCP header location and size. We can use Property.TCPPayloadLength to obtain the total length of the payload. And to get the offset into the frame we use the TCP header length (TCP.DataOffset.DataOffset). Finally to get the start of the TCP frame we use the offset of TCP.SrcPort which is the first field in a TCP frame. With these pieces of information, we can use NmGetPartialRawFrame API to grab the raw data from the frame.

So here's the code snippet:

void
GetFramePayload(HANDLE ParsedFrame, HANDLE FrameParser, HANDLE RawFrame)
{
ULONG ret;
UINT32 PayloadLen = 0;
ULONG retlen;
NmPropertyValueType PropType;

UINT8 TCPHeaderSize;
ULONG TCPSrcOffset, TCPSrcSize;


// Get Payload Length
ret = NmGetPropertyValueById(FrameParser, TCPPayloadLengthID, sizeof(PayloadLen), (PBYTE)&PayloadLen, &retlen, &PropType);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Error retrieving TCP Payload Length Property, err=%d\n", ret);
return;
}

if(PayloadLen > 0)
{
// Get the Data Offset, used to determine the TCP header size
ret = NmGetFieldValueNumber8Bit(ParsedFrame, TCPDataOffsetID, &TCPHeaderSize);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Error retrieving TCP Header Length Field, err=%d\n", ret);
return;
}

// Get the Offset of TCP.SrcPort which is the first field in TCP.
ret = NmGetFieldOffsetAndSize(ParsedFrame, TCPSrcPortID, &TCPSrcOffset, &TCPSrcSize);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Error retrieving TCP SRC Header/Offset, err=%d\n", ret);
return;
}

wprintf(L"Offset: %d, Length: %d, HeaderLen: %d\n", TCPSrcOffset/8, PayloadLen, TCPHeaderSize*4);

// Allocate a buffer based on the Payload Length Property.
PBYTE buf = (PBYTE)malloc(PayloadLen);

// Read in the partial frame. The Offset is in bits. TCPHeaderSize is off by a factor of 4.
ret = NmGetPartialRawFrame(RawFrame, TCPSrcOffset/8 + TCPHeaderSize*4, PayloadLen, buf, &retlen);

// Do what ever you want with buf now. I'll assume it's ASCII and print it.
wprintf(L"%S", buf);
}
}

And here is the initialization code for each of our frame parser to see how each data field and property was added:

HANDLE
MyLoadNPL(void)
{
HANDLE myFrameParser = INVALID_HANDLE_VALUE;
ULONG ret;

// Use NULL to load default NPL set.
ret = NmLoadNplParser(NULL, NmAppendRegisteredNplSets, MyParserBuild, 0, &NplParser);

if(ret == ERROR_SUCCESS){
ret = NmCreateFrameParserConfiguration(NplParser, MyParserBuild, 0, &FrameParserConfig);

if(ret == ERROR_SUCCESS)
{

ret = NmAddProperty(FrameParserConfig, L"Property.TCPPayloadLength", &TCPPayloadLengthID);
if(ret != 0)
{
wprintf(L"Failed to add Property.TCPPayloadLength, error 0x%X\n", ret);
}

ret = NmAddField(FrameParserConfig, L"TCP.SrcPort", &TCPSrcPortID);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Failed to add field, TCP.SrcPort, error 0x%X\n", ret);
}

ret = NmAddField(FrameParserConfig, L"TCP.DataOffset.DataOffset", &TCPDataOffsetID);
if(ret != ERROR_SUCCESS)
{
wprintf(L"Failed to add field, TCP.DataOffset, error 0x%X\n", ret);
}

ret = NmCreateFrameParser(FrameParserConfig, &myFrameParser);

if(ret != ERROR_SUCCESS)
{
wprintf(L"Failed to create frame parser, error 0x%X\n", ret);
NmCloseHandle(FrameParserConfig);
NmCloseHandle(NplParser);
return INVALID_HANDLE_VALUE;
}
}
else
{
wprintf(L"Unable to load parser config, error 0x%X\n", ret);
NmCloseHandle(NplParser);
return INVALID_HANDLE_VALUE;
}

}
else
{
wprintf(L"Unable to load NPL\n");
return INVALID_HANDLE_VALUE;
}

return(myFrameParser);
}

By using TCP.SrcPort, we get rid of any dependency of the stack. This will work on IPv4, IPv6 or any tunneled protocols. Also the TCP.PayloadLength is computed by the parsers which again is agnostic to the carrying protocols.

Party on Your Payload

Now that you have your payload in a BYTE buffer, you can do what ever you want with it. For instance, if you wanted to create an expert to show each payload and response as text, you could simply take the frame number that is referenced and use that to determine the conversation key for the TCP conversation, i.e. using a property Conversation.ID.TCP. Then you can use this to filter all other packets in the same trace with the same TCP Conversation ID. This would give you a high level view of text based traffic like HTTP and FTP. Of course there is a little more work to deal with fragmented data, but the API gives you all the tools to accomplish this.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • I'm developing an IDS program that can analyze the payload, if you can help me?

  • Please feel free to post a question in our forums.  We can help you out there.

    http://social.technet.microsoft.com/Forums/en-US/netmon/threads

  • hi

    i need help...

    i am a 3rd yr student n doing a project of network monitoring in c#...

    i am capturing data over a network bt not able to see wat the data is...i need to decrypt it bt dnt knw how....i have all the information abt the header bt no information abt the data...plz plz plz help....its really very urgent...i hav project evaluation...plz help....

  • Are you looking at SSL/TLS traffic?  If so we have an expert on http://nmdecrypt.codeplex.com which has some code to show you how to decrypt the data.  I suppose you could also just call out to the expert since it's a separate EXE with command line arguments.  

    If you need more information, please feel free to ask on our forums: social.technet.microsoft.com/.../threads

  • I Like the RHCP reference "Party on your 'Payload'"