Message tracking versions
· Exchange 2003
o Structure is different, still csv but fields are not compatible to Exchange 2007 or Exchange 2010
o Message tracking will stop at last Exchange 2003 Server.
· Exchange 2007
o Message tracking from 2003/2010 will stop. Exchange 2007 tracking will only track on 2007 servers. The tracking fields are different.
· Exchange 2010
o Message tracking from 2003/2007 will stop. Exchange 2010 tracking will only be track on 2010 servers…the tracking fields are different.
Exchange Tracking logs fields / events
Exchange 2003 *See Appendix I
· http://support.microsoft.com/kb/246965
· http://support.microsoft.com/kb/821905
Exchange 2007 *See Appendix II
· http://technet.microsoft.com/en-us/library/cc539064.aspx
· http://technet.microsoft.com/en-us/library/cc539063.aspx
Exchange 2010 *See Appendix III
· This is really a reference to Exchange 2007 since the Exchange 2010 is NOT updated note there are some differences.
· http://technet.microsoft.com/en-us/library/bb124926(EXCHG.80).aspx
How to Track a Message via PowerShell effectively
Get-MessageTrackingLogs
So the best way to track a message is via its Message ID. You can retrieve this from the NDR or you can Search by the Sender / Recipient but note this will get all messages that recipient received or sender sent.
Now here is an issue with Exchange 2010 / Outlook 2010, the message in the Sent Item does not have a Message ID as that is assigned once its hits the Hub. L
So find the Message ID you need the Sender and Recipient Addresses and Subject and or Time will be Nice if they are sending a lot of messages between them…
1. Try with Sender/Recipient/Subject and Time (Give yourself a pretty good range). Now what the GUI does for you is filter on the Receive Event, but you can do it this way as well. Be careful of or event orders…you should sort by TimeStamp as this to the powershel. “|sort –Property TimeStamp” before any “|Fl”
Get-MessageTrackingLog –Sender “Sender@domain.com” –Recipients “Recipient@domain.com” -MessageSubject “Subject of Message” -Start "3/28/2011 8:00AM" -End "3/28/2011 5:00PM"
NOW be careful! In this example this is a Single Message caught. That will not always be the case.
2. Next, you Add the | FL Sender, Recipients, MessageSubject, MessageId
Get-MessageTrackingLog –Sender “Sender@domain.com” –Recipients “Recipient@domain.com” -MessageSubject “Subject of Message” -Start "3/28/2011 8:00AM" -End "3/28/2011 5:00PM" | FL Sender, Recipients, MessageSubject, MessageId
Now look at the MessageID and then we will track the message via MessageID. This is the BEST way to track a message.
3. Now, add the –MessageID “MessageID” and change the |FL * (this gets all fields)
Get-MessageTrackingLog –Sender “Sender@domain.com” –Recipients “Recipient@domain.com” -MessageSubject “Subject of Message” -Start "3/28/2011 8:00AM" -End "3/28/2011 5:00PM" –MessageID “MessageID” | FL *
Hint: PSComputerName is the top of each entry.
4. Ok, now that we got this what does it mean?
a. Since ClientHostName and ServerHostName that is telling me the message was sent from this server to itself. (this was down with a Powershell SMTP Send Script). So typically the ClientHostName will be the Remote Server Submitting the message.
b. Since Source is SMTP, and the Event is Receive. That means the ClientHostName Submitted the message via SMTP to the ServerHostName.
c. EventData I can also tell this was the First hop in the Org.
d. Ok, I receive it what next…
e. Notice the ClientHostName and the ServerHost name are not changed. Client is not the CAS server and the Server is the Mailbox Role Server
f. Since the Source is StoreDriver and Event is Delivery, the message was submitted to the store.
First PowerShell Output….
[PS] C:\>Get-MessageTrackingLog -Sender User19-DB01@TailSpinToys.com -Recipients User1-DB01@TailSpinToys.com -MessageSubject "Origins of Legislation" -Start "4/19/2011 4:43:30 PM" -End "4/19/2011 4:50:30 PM" -MessageID "1cd4eba2-d158-4ea1-81a7-4dbbc659bd13@LAB-E2K10CSHT01.TailSpinToys.com" |fl *
PSComputerName : lab-e2k10csht01.tailspintoys.com
RunspaceId : a241bdf5-c2c6-4c99-8e5a-b395faa5e67a
Timestamp : 4/19/2011 4:45:30 PM
ClientIp : fe80::89dc:2ad8:e3b:c03%13
ClientHostname : LAB-E2k10CSHT01
ServerIp : fe80::89dc:2ad8:e3b:c03%13
ServerHostname : LAB-E2k10CSHT01
SourceContext : 08CDCCED60881B31;2011-04-19T21:45:30.419Z;0
ConnectorId : LAB-E2K10CSHT01\Default LAB-E2K10CSHT01
Source : SMTP
EventId : RECEIVE
InternalMessageId : 270
MessageId : <1cd4eba2-d158-4ea1-81a7-4dbbc659bd13@LAB-E2K10CSHT01.TailSpinToys.com>
Recipients : {User1-DB01@TailSpinToys.com}
RecipientStatus : {}
TotalBytes : 4146
RecipientCount : 1
RelatedRecipientAddress :
Reference :
MessageSubject : Origins of Legislation
Sender : User19-DB01@TailSpinToys.com
ReturnPath : User19-DB01@TailSpinToys.com
MessageInfo : 0aI: NTS:
MessageLatency :
MessageLatencyType : None
EventData : {[FirstForestHop, LAB-E2K10CSHT01.TailSpinToys.com]}
ClientIp :
ServerIp :
ServerHostname : LAB-E2K10MBX02
SourceContext : 08CDCCED60881B32;2011-04-19T21:45:30.575Z;0
ConnectorId :
Source : STOREDRIVER
EventId : DELIVER
TotalBytes : 4318
MessageInfo : 2011-04-19T21:45:30.450Z;SRV=LAB-E2K10CSHT01.TailSpinToys.com:TOTAL=0
MessageLatency : 00:00:00.2970000
MessageLatencyType : EndToEnd
EventData : {[MailboxDatabaseName, e2k10db01]}
Same Example Pulled from Excel to Compare Fields
#Fields:
date-time 2011-04-19T21:45:30.560Z
client-ip fe80::89dc:2ad8:e3b:c03%13
client-hostname LAB-E2k10CSHT01
server-ip fe80::89dc:2ad8:e3b:c03%13
server-hostname LAB-E2k10CSHT01
source-context 08CDCCED60881B31;2011-04-19T21:45:30.419Z;0
connector-id LAB-E2K10CSHT01\Default LAB-E2K10CSHT01
source SMTP
event-id RECEIVE
internal-message-id 270
message-id <1cd4eba2-d158-4ea1-81a7-4dbbc659bd13@LAB-E2K10CSHT01.TailSpinToys.com>
recipient-address User1-DB01@TailSpinToys.com
recipient-status
total-bytes 4146
recipient-count 1
related-recipient
-address
reference
message-subject Origins of Legislation
sender-address User19-DB01@TailSpinToys.com
return-path User19-DB01@TailSpinToys.com
message-info 0aI: NTS:
directionality Originating
tenant-id
original-client-ip fe80::89dc:2ad8:e3b:c03%13
original-server-ip fe80::89dc:2ad8:e3b:c03%13
custom-data S:FirstForestHop=LAB-E2K10CSHT01.TailSpinToys.com
date-time 2011-04-19T21:45:30.747Z
client-ip
server-ip
server-hostname LAB-E2K10MBX02
source-context 08CDCCED60881B32;2011-04-19T21:45:30.575Z;0
connector-id
source STOREDRIVER
event-id DELIVER
total-bytes 4318
message-info 2011-04-19T21:45:30.450Z;SRV=LAB-E2K10CSHT01.TailSpinToys.com:TOTAL=0
original-client-ip
original-server-ip
custom-data S:MailboxDatabaseName=e2k10db01
Compare table: Green only from Logs, Blue only from PwShell
PWShell
Logs
Actual Data
Timestamp
date-time
2011-04-19T21:45:30.560Z
ClientIp
fe80::89dc:2ad8:e3b:c03%13
ClientHostname
client-hostname
LAB-E2k10CSHT01
ServerIp
ServerHostname
server-hostname
SourceContext
source-context
08CDCCED60881B31;2011-04-19T21:45:30.419Z;0
ConnectorId
LAB-E2K10CSHT01\Default LAB-E2K10CSHT01
Source
source
SMTP
EventId
event-id
RECEIVE
InternalMessageId
internal-message-id
270
MessageId
message-id
<1cd4eba2-d158-4ea1-81a7-4dbbc659bd13@LAB-E2K10CSHT01.TailSpinToys.com>
Recipients
recipient-address
User1-DB01@TailSpinToys.com
RecipientStatus
{}
TotalBytes
total-bytes
4146
RecipientCount
recipient-count
1
RelatedRecipientAddress
related-recipient-address
Reference
MessageSubject
message-subject
Origins of Legislation
Sender
sender-address
User19-DB01@TailSpinToys.com
ReturnPath
return-path
MessageInfo
message-info
0aI: NTS:
directionality
Originating
custom-data
MessageLatency
MessageLatencyType
None
EventData
{[FirstForestHop, LAB-E2K10CSHT01.TailSpinToys.com]}
Advanced PowerShell Examples
Get-ExchangeServer | where {$_.isHubTransportServer -eq $true -or $_.isMailboxServer -eq $true} | Get-MessageTrackingLog -MessageId "<messageid>" | Select-Object <commaseparatedfieldnames> | Sort-Object -Property <field>
Get-ExchangeServer | where {$_.isHubTransportServer -eq $true -or $_.isMailboxServer -eq $true} | Get-MessageTrackingLog -MessageId "ba18339e-8151-4ff3-aeea-87ccf5fc9796@contoso.com" | Select-Object Timestamp,ServerHostname,ClientHostname,Source,EventId,Recipients | Sort-Object -Property Timestamp
What does this do? It searches every Exchange Server (Hub and Mailbox) for the Message ID listed, selects specific attributes and then Sorts them…
In this case, it dumps out the Timestamp,ServerHostname,ClientHostname,Source,EventId,Recipients in a nice easy to read format…
[PS] C:\Windows\system32>Get-ExchangeServer | where {$_.isHubTransportServer -eq $true -or $_.isMailboxServer -eq $true}
| Get-MessageTrackingLog -MessageId "1cd4eba2-d158-4ea1-81a7-4dbbc659bd13@LAB-E2K10CSHT01.TailSpinToys.com" | Select-Ob
ject Timestamp,ServerHostname,ClientHostname,Source,EventId,Recipients | Sort-Object -Property Timestamp
[PS] C:\Windows\system32>
Using ConvertTo-MessageLatency.ps1 with Get-MessageTrackingLog!
[PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>Get-MessageTrackingLog -MessageId "1cd4eba2-d158-4ea1-81a7-4
dbbc659bd13@LAB-E2K10CSHT01.TailSpinToys.com" | .\ConvertTo-MessageLatency.ps1
ComponentServerFqdn : LAB-E2K10CSHT01.TailSpinToys.com
ComponentCode : TOTAL
ComponentName : Total Server Latency
ComponentLatency : 00:00:00
[PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>
Appendix I
Exchange 2003
Fields:
Field number
Field name
Description
Date
Date of the event.
2
Time
Greenwich mean time of the event.
3
Client-IP
IP of connecting client.
4
Client-hostname
Hostname of connecting client.
5
Partner-name
Name of the messaging service that the message is handed off to. In Exchange 2000, the service can be: SMTP, X400, MAPI, IMAP4, POP3, STORE. This is essentially the same as Exchange Server 5.5, but in Exchange 2000, there are more possibilities for this field.
6
Server-hostname
Hostname of the server that is making the log entry.
7
Server-IP
IP of the server that is making the log entry.
8
Recipient-address
Message recipient (SMTP or X.400 address).
9
Event-ID
Integer corresponding to the Event ID of the action logged, for example: sent, received, delete, retrieve.
10
MSGID
Message ID.
11
Priority
The priority is represented by -1 if low, 0 if normal, 1 if high
12
Recipient-Report-Status
A number representing the result of an attempt to deliver a report to the recipient: 0 if delivered, 1 if not delivered. This is used only for reports (non-delivery reports [NDRs], delivery receipts [DRs]). On other events, it is blank.
13
Total-bytes
Message size (in bytes).
14
Number-recipients
Total number of recipients.
15
Origination-time
Delivery time (in seconds) representing the time it takes to deliver the message. Determined from the difference between the timestamp and time encoded in Message ID. Only valid for messages within the Exchange organization (all versions); there is no requirement to decode other product message IDs such as Sendmail, and so on.
16
Encryption
For the primary body part: 0 if no encryption, 1 if signed only, 2 if encrypted. This is per message, not per recipient.
17
Service-version
Version of the service making the log entry.
18
Linked-MSGID
If there is a MSG ID from another service, it is given here to link the message across services.
19
Message-subject
The subject of the message, truncated to 256 bytes.
20
Sender-address
Primary address of the originating mailbox, if known. This could be SMTP, X.400, or Distinguished Name (DN), depending on transport
Event-ID in Field 9:
Event Number
Event Type
0
Message transfer in
The message was received from a server, a connector, or a gateway.
Probe transfer in
An X.400 probe was received from a gateway, a link, or a message transfer agent (MTA).
Report transfer in
A delivery receipt or a non-delivery report (NDR) was received from a server, a connector, or a gateway.
Message submission
The message was sent by the client.
Probe submission
An X.400 probe was received from a user.
Probe transfer out
An X.400 probe was sent to a gateway, a link, or an MTA.
Message transfer out
The message was sent to a server, a connector, or a gateway.
Report transfer out
A delivery receipt or an NDR was sent to a server, a connector, or a gateway.
Message delivered
The message was delivered to a mailbox or a public folder.
Report delivered
A delivery receipt or an NDR was delivered to a mailbox.
StartAssocByMTSUser
23
ReleaseAssocByMTSUse
28
Message redirected
The message was sent to mailboxes other than the mailboxes of the recipients.
29
Message rerouted
The message was routed to an alternative path.
31
Downgrading
An X.400 message was downgraded to 1984 format before relay.
33
Report absorption
The number of delivery receipts or of NDRs exceeded a threshold and the reports were deleted.
34
Report generation
A delivery receipt or an NDR was created.
43
Unroutable report discarded
A delivery receipt or an NDR could not be routed and was deleted from the queue.
50
Gateway deleted message
The administrator deleted an X.400 message that was queued for a gateway.
51
Gateway deleted probe
The administrator deleted an X.400 probe that was queued for a gateway.
52
Gateway deleted report
The administrator deleted an X.400 report that was queued for a gateway.
1000
Local delivery
The sender and the recipient are on the same server.
1001
Backbone transfer in
Mail was received from another MAPI system across a connector or across a gateway.
1002
Backbone transfer out
Mail was sent to another MAPI system across a connector or across a gateway.
1003
Gateway transfer out
The message was sent through a gateway.
1004
Gateway transfer in
The message was received from a gateway.
1005
Gateway report transfer in
A delivery receipt or an NDR was received from a gateway.
1006
Gateway report transfer out
A delivery receipt or an NDR was sent through a gateway.
1007
Gateway report generation
A gateway generated an NDR for a message.
1010
SMTP queued outbound
Outgoing mail was queued for delivery by the Internet Mail Service.
1011
SMTP transferred outbound
Outgoing mail was transferred to an Internet recipient.
1012
SMTP received inbound
Incoming mail was received from by the Internet Mail Service.
1013
SMTP transferred
Incoming mail that was received by the Internet Mail Service was transferred to the information store.
1014
SMTP message rerouted
An Internet message is being rerouted or forwarded to the correct location.
1015
SMTP report transferred In
A delivery receipt or an NDR was received by the Internet Mail Service
1016
SMTP report transferred out
A delivery receipt or an NDR was sent to the Internet Mail Service.
1017
SMTP report generated
1018
SMTP report absorbed
The receipt or the NDR could not be delivered and was absorbed. (You cannot send an NDR for an NDR.)
1019
SMTP submit message to AQ
A new message is submitted to Advanced Queuing.
1020
SMTP begin outbound transfer
A message is about to be sent over the wire by SMTP.
1021
SMTP bad mail
The message was transferred to the Badmail folder.
1022
SMTP AQ failure
A fatal Advanced Queuing error occurred. Information about the failure was written to the Event Manager.
1023
SMTP local delivery
A message was successfully delivered by a store drive (logged by Advanced Queue).
1024
SMTP submit message to cat
Advanced Queuing submitted a message to the categorizer.
1025
SMTP begin submit message
A new message was submitted to Advanced Queuing.
1026
SMTP AQ failed message
Advanced Queuing could not process the message. The message caused an NDR to be sent, or the message was put in the Badmail folder.
1027
SMTP submit message to SD
A message was submitted to the store driver by the MTA.
1028
SMTP SD local delivery
The store driver successfully delivered a message (logged by store driver).
1029
SMTP SD gateway delivery
The store driver transferred the message to the MTA.
1030
SMTP NDR all
All recipients were sent an NDR.
1031
SMTP end outbound transfer
The outgoing message was successfully transferred.
1032
SMTP message scheduled to retry categorization
1033
SMTP message categorized and queued for routing
1034
SMTP message routed and queued for remote delivery
1035
SMTP message scheduled to retry routing
1036
SMTP message queued for local delivery
1037
SMTP message scheduled to retry local delivery
1038
SMTP message routed and queued for gateway delivery
1039
SMTP message deleted by Intelligent Message Filtering
1040
SMTP message rejected by Intelligent Message Filtering
1041
SMTP message archived by Intelligent Message Filtering
1042
Message redirected to the alternate recipient
Appendix II
Fields that are marked with an asterisk (*) are never blank.
date-time*
The date and time of the message tracking event. The value is formatted as yyyy-mm-ddhh:mm:ss.fffZ, where yyyy = year, mm = month, dd = day, hh = hour, mm = minute, ss = second, fff = fractions of a second, and Z signifies Zulu, which is another way to denote UTC.
The TCP/IP address of the messaging server or messaging client that submitted the message.
The name of the messaging server or messaging client that submitted the message.
The TCP/IP address of the source or destination server running Microsoft Exchange Server.
The name of the destination server.
Extra information associated with the source field.
The name of source or destination Send connector or Receive connector.
source*
The Exchange transport component responsible for the message tracking event. The possible values for this field are as follows:
· ADMIN (for Replay directory submission)
· AGENT
· DSN
· GATEWAY (for Foreign connector submission)
· PICKUP
· ROUTING
· SMTP
· STOREDRIVER
event-id*
The message event type. These events are described fully in the table earlier in this topic. The possible values are BADMAIL, DELIVER, DSN, EXPAND, FAIL, POISONMESSAGE, RECEIVE, REDIRECT, RESOLVE, SEND, SUBMIT, and TRANSFER.
internal-message-id*
A message identifier that is assigned by the Exchange Server 2007 server that is currently processing the message. A specific message's value of internal-message-id is different in the message tracking log of every Exchange Server 2007 server that is involved in the delivery of the message.
The value of the Message-Id: field found in the message's header fields. If the Message-Id: header field does not exist or is blank, an arbitrary value is assigned. This value is constant for the lifetime of the message.
recipient-address*
A message was submitted by a server running Exchange Server 2007 computer that has the Mailbox server role installed to an Exchange 2007 computer that has the Hub Transport server role or Edge Transport server role installed.
The e-mail addresses of the message's recipients. Multiple e-mail addresses are separated by the semicolon character (;).
total-bytes*
The number of recipients in the message.
recipient-count*
This field is used with EXPAND, REDIRECT, and RESOLVE events to display other recipient e-mail addresses associated with the message.
This field contains additional information for specific types of events:
· DSN - The Reference field contains the Internet-Message-Id of the message that caused the DSN.
· SEND - The Reference field contains the Internet-Message-Id of any delivery status notification (DSN) messages.
· TRANSFER - The Reference field contains the Internal-Message-Id of the message that is being forked.
For all other types of events, the Reference field is blank.
The message's subject found in the Subject: header field. The tracking of message subjects is controlled by the MessageTrackingLogSubjectLoggingEnabled parameter in the Set-TransportServer cmdlet for Hub Transport servers and Edge Transport servers, or in the Set-MailboxServer cmdlet for Mailbox servers. By default, message subject tracking is enabled. Message subject logging can be disabled by setting the value of the MessageTrackingLogSubjectLoggingEnabled parameter to $false.
The e-mail address specified in the Sender: header field, or the From: header field if Sender: is not present.
return-path*
The return e-mail address specified by MAIL FROM: in the message envelope. Although this field is never empty, it can have the null sender address value represented as <>.
This field contains the message origination date-time for DELIVER and SEND events. The origination date-time is the time that the message first enters the Exchange organization. The value is formatted as yyyy-mm-ddhh:mm:ss.fffZ, where yyyy = year, mm = month, dd = day, hh = hour, mm = minute, ss = second, fff = fractions of a second, and Z signifies Zulu, which is another way to denote UTC.
Table: Event Types Used to Classify Each Message Event
These are the Event-ID field above
Event name
BADMAIL
A message was submitted by the Pickup directory or the Replay directory that cannot be delivered or returned.
DELIVER
A message was delivered to a mailbox.
DSN
A delivery status notification (DSN) was generated.
EXPAND
A distribution group was expanded.
FAIL
A message delivery failed.
POISONMESSAGE
A message is put in the poison message queue or removed from the poison message queue.
SUSPEND
Indicates that replication has been halted for the passive copy. This state prevents the database from advancing, and logs from being copied. Possible values are True and False.
A message was received and committed to the database.
REDIRECT
A message was redirected to an alternative recipient after an Active Directory directory service lookup.
RESOLVE
A message's recipients were resolved to a different e-mail address after an Active Directory lookup.
SEND
A message was sent by Simple Mail Transfer Protocol (SMTP) to a different server.
SUBMIT
A message was submitted by an Exchange Server 2007 computer that has the Mailbox server role installed to an Exchange Server 2007 computer that has the Hub Transport server role or Edge Transport server role installed.
TRANSFER
Recipients were moved to a forked message because of content conversion, message recipient limits, or agents.
Comparing the field names that are used in the message tracking log and the field names that are used by the Get-MessageTrackingLog cmdlet
Field name that is used in the message tracking log
Field name that is used to filter the Get-MessageTrackingLog results
Appendix III
Search filters that are available by using the Get-MessageTrackingLog cmdlet
Search filter
Corresponding field in the message tracking log
End
ResultSize
None. This parameter limits the number of results that are displayed by the search.
Start
Queues
· Get-Queue
· Resume-Queue
· Retry-Queue
· Suspend-Queue
Messages
· Get-Message
· Remove-Message
· Resume-Message
· Suspend-Message
· Export-Messages
GUI is BAD! Cmd line good!
So the First thing we get a SCOM alert mentioning a Queue is over its threshold on ServerX.
1. In Exchange Shell do Get-Queue –Server <servername> and see what other queues may be affected.
Get-Queue –Server <ServerName>
[PS] C:\scripts>get-Queue -Server Lab-E2k10CSHT01
Identity DeliveryType Status MessageCount NextHopDomain
-------- ------------ ------ ------------ -------------
LAB-E2k10CSHT01\4 SmtpRelay... Retry 2196 adsite2
LAB-E2k10CSHT01\Submission Undefined Ready 0 Submission
2. Notice the NextHopDomain, this will show the next location the message is to be routed. The adsite2 is for a mailbox in the remote ADSite2. This can be another Queue or a SMTP Domain.
3. Let’s look at this closer, add an “|FL” to the command and see what information is added. The Key ones are the LastError, Status, LastRetryTime, NextretryTime. The lastError in this example not only gives the SMTP code "421 4.2.1 Unable to connect." But also elaborates to the cause
[PS] C:\scripts>get-Queue -Server Lab-E2k10CSHT01 |fl
RunspaceId : 59f16a72-c038-4c79-9c48-d1b14d5411bf
DeliveryType : SmtpRelayToRemoteAdSite
NextHopDomain : adsite2
TlsDomain :
NextHopConnector : 2c2bba82-cdb3-460a-82b3-6816d7e7fb70
Status : Retry
MessageCount : 2196
LastError : 451 4.4.0 Primary target IP address responded with: "421 4.2.1 Unable to connect." Attempted faillover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts.
LastRetryTime : 4/22/2011 8:57:34 AM
NextRetryTime : 4/22/2011 9:07:34 AM
DeferredMessageCount : 0
QueueIdentity : LAB-E2k10CSHT01\4
Identity : LAB-E2k10CSHT01\4
IsValid : True
DeliveryType : Undefined
NextHopDomain : Submission
NextHopConnector : 00000000-0000-0000-0000-000000000000
Status : Ready
MessageCount : 0
LastError :
LastRetryTime :
NextRetryTime :
QueueIdentity : LAB-E2k10CSHT01\Submission
Identity : LAB-E2k10CSHT01\Submission
4. Note: You could alternately use get-Queue –Identity “<Queue>” |fl
[PS] C:\scripts>get-Queue -Identity "LAB-E2k10CSHT01\4" |fl
LastError : 451 4.4.0 Primary target IP address responded with: "421 4.2.1 Unable to connect." Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts.
5. Ok, what do I do? Well, that depends on the error. In this case, I just need to start the Hub Transport in my Remote Site. That LastError message is the key to troubleshooting the reason for the Queue backup.
a. Remote Queues will all be SMTP Based. SMTP based Queues use the SMTP Error Codes. *See the SMTP Troubleshooting See http://support.microsoft.com/kb/256321/
i. Note: These will vary in appearance, name and number.
b. Mailbox or Local Submission queues check Database for Errors/warning
c. “Submission” will always be AV or an Agent/Rule triggered on Submission Event.
d. Poison Message Queue – These are messages that have caused the Transport Service to crash.
e. Shadow Queues will always be that something is preventing the Q-Discard from being received / sent.
f. Unreachable Queue – anytime there is not a route to the destination. Check Config to these recipients.
6. Now, here is some helpful “-Filter” to help with looking at Queues. Note Get-Queue only gets the queue on the local server. You will have to use Get-ExchangeServer | Where { $_.ServerRole –like “*transport*”} or Get-ExchangeServer | where {$_.isHubTransportServer -eq $true } or something similar to look at all Server Queues.
a. To get all the queues with MessageCount Greater Than 100
Get-queue –Filter {MessageCount –gt 200}
b. To get all the queues with LastError not equal to $Null
Get-Queue –Filter {LastError –ne $Null}
c. And the list can go on… Common Comparison Operators
Operator
Name
-eq
Equality
Tests whether values are the same. Other languages may use = or == to test for equality.
-ne
Not Equal
Tests for inequality. Other languages may use <> or != to test for inequality.
-gt
Greater Than
Tests whether one value is larger than another. Other languages may use the > character.
-lt
Less Than
Tests whether one value is smaller than another. Other languages may use the < character.
-ge
Greater Than or Equal To
Tests whether a value is larger than or equal to another value. Similar to >= in VBScript and other languages.
-le
Less Than or Equal To
Tests whether a value is smaller than or equal to another value. Similar to <= in VBScript and other languages.
7. Ok, I found the Queue and I want to Export the messages…how? Well, I am glad you asked. Use the Export-message command. Note it is best to Suspend the Queue or Message before exporting.
a. Now, there are several options.
i. Export a Specific Message
Export-Message -Identity LAB-E2k10CSHT01\4 -Path "C:\Tailspintoys\export.eml"
ii. Export ALL message from a Specific Queue, this gets a little more involved. Requires a ForEach loop and some logic for the name of the files exported is highlighted.
Get-Message -Queue "LAB-E2k10CSHT01\4" | ForEach-Object
{
$Temp="C:\tailspintoys Export\"+$_.InternetMessageID+".eml";
$Temp=$Temp.Replace("<","_");
$Temp=$Temp.Replace(">","_");
Export-Message $_.Identity | AssembleMessage -Path $Temp
}
iii. Export ALL Message from a Server
Get-Message -Server "LAB-E2k10CSHT01" | ForEach-Object
iv. Now, you can add filter to the above say you only wanted to get-messages from senders in the Tailspintoys domain. You could add -Filter {FromAddress -like "@Tailspintoys.com"} to the above after Get-Message.
v. To Replay these messages on any Exchange 2010 Hub Server, drop into the “Replay” Directory.
8. Ok, how do I delete them? With and Without an NDR….well the Remove message does both! Only difference in the command –WithNDR $True or $False
a. Remove-Message -Filter {property -operator "value"} -WithNDR <$true | $false>
Remove-Message -Filter {Subject -eq "Win Big"} -WithNDR $false will Turf the message withOUT NDRs being sent.
Remove-Message -Filter {Subject -eq "Win Big"} -WithNDR $True will Turf the message with an NDR. Since the default is True. You could remove the –WithNDR $True
b. The –Filter is the same that you used with the Get-Message.
· DeliveryType The delivery type for this queue as defined by transport. The delivery type must be one of the following values:
o DNSConnectorDelivery
o NonSMTPGatewayDelivery
o SmartHostConnectorDelivery
o SmtpRelayWithinAdSitetoEdge
o MapiDelivery
o SmtpRelayWithinAdSite
o SmtpRelaytoRemoteAdSite
o SmtpRelaytoTiRg
o Undefined
o Unreachable
· Identity The queue identity in the form of Server\destination, where destination is a remote domain, Mailbox server, or persistent queue name.
· LastError A text string of the last error recorded for a queue.
· LastRetryTime The time when a connection was last tried for this queue.
· MessageCount The number of items in the queue.
· NextHopConnector The GUID of the connector that was used to create the queue.
· NextHopDomain The next hop domain of the queue, specified as a remote SMTP domain, a server name, the name of an Active Directory site, or a message database (MDB) identifier.
· NextRetryTime The time when a connection will next be tried for this queue.
· Status The status of the queue. Queue status options are Active, Ready, Retry, or Suspended.
Queue identity formats
Queue identity format
Usage
Server\QueueJetID (Int64)
The complete, unique identity for a delivery queue.
\QueueJetID
The identity of a queue on the local server. The server name is omitted. Therefore, the local server is implied.
Server\*
Any queue on the specified server.
Server\NextHopDomain
A queue on the specified server holding messages for delivery to a specific remote domain.
\NextHopDomain
A queue holding messages destined for a specific domain and located on the local server.
Server\Poison
The poison message queue located on the specified server.
Server\Submission
The queue that contains items waiting to be processed by the categorizer.
Server\Unreachable
The queue that contains items that can't be routed and located on the specified server.
Message Filters
The Filter parameter requires an expression that specifies the property value criteria for the messages that you want to remove. The expression includes a property name followed by a comparison operator and value. The following message properties are valid criteria for the Filter parameter:
· DateReceived The date that the message was received.
· ExpirationTime The time that a message will expire.
· FromAddress The SMTP address of the sender of a message. This value is taken from MAIL FROM in the message envelope.
· Identity An integer that represents a particular message and an optional server and queue identity.
· InternetMessageId The value of the Message-ID header field. This property is expressed as a GUID followed by the SMTP address of the sending server, as in this example: 67D7543D6103DC4FBEBA6BC7205DACABA61231@exchange.contoso.com.
· LastError A text string of the last error recorded for a message.
· MessageSourceName A text string of the name of the component that submitted this message to the queue.
· Queue The identity of the queue that holds the message. Enter the queue identity in the form of Server\destination, where destination is a remote domain, Mailbox server, or persistent queue name.
· RetryCount The number of times that delivery of a message to a destination was tried.
· SCL The spam confidence level (SCL) of the message. Valid SCL entries are integers 0 through 9. An empty SCL property value indicates that the message hasn't been processed by the Content Filter agent.
· Size The size of a message.
· SourceIP The IP address of the external server that submitted the message to the Exchange organization.
· Status The current message status. Message status options are Active, Retry, Suspended, PendingSuspend, and PendingRemove.
· Subject A text string that represents the e-mail subject. The value is taken from the Subject header field.
You can specify multiple filter criteria by using the and comparison operator. Property values that aren't expressed as a single integer must be enclosed in quotation marks (").
Comparison Operators.
TOPIC
about_Comparison_Operators
SHORT DESCRIPTION
Describes the operators that compare values in Windows PowerShell.
LONG DESCRIPTION
Comparison operators let you specify conditions for comparing values and
finding values that match specified patterns. To use a comparison operator,
specify the values that you want to compare together with an operator that
separates these values.
By default, all comparison operators are case-insensitive. To make a
comparison operator case-sensitive, precede the operator name with a "c".
For example, the case-sensitive version of "-eq" is "-ceq". To make the
case-insensitivity explicit, precede the operator with an "i". For example,
the explicitly case-insensitive version of "-eq" is "ieq".
All comparison operators except the containment operators
(-contains, -notcontains) and type operators (-is, -isnot) return a Boolean
value when the input to the operator (the value on the left side of the
operator) is a single value (a scalar). When the input is a collection of
values, the containment operators and the type operators return any
matching values. If there are no matches in a collection, these operators
do not return anything. The containment operators and type operators always
return a Boolean value.
Windows PowerShell supports the following comparison operators.
Description: Equal to. Includes an identical value.
Example:
C:\PS> "abc", "def" -eq "abc"
abc
Description: Not equal to. Includes a different value.
C:\PS> "abc", "def" -ne "abc"
def
Description: Greater-than.
C:\PS> 8 -gt 6
True
Description: Greater-than or equal to.
C:\PS> 8 -ge 8
Description: Less-than.
C:\PS> 8 -lt 6
False
Description: Less-than or equal to.
C:\PS> 6 -le 8
-like
Description: Match using the wildcard character (*).
C:\PS> "Windows PowerShell" -like "*shell"
-notlike
Description: Does not match using the wildcard character (*).
C:\PS> "Windows PowerShell" -notlike "*shell"
-match
Description: Matches a string using regular expressions.
When the input is scalar, it populates the
$Matches automatic variable.
C:\PS> "Sunday" -match "sun"
C:\PS> $matches
Name Value
---- -----
0 sun
-notmatch
Description: Does not match a string. Uses regular expressions.
When the input is scalar, it populates the $Matches
automatic variable.
C:\PS> "Sunday" -notmatch "sun"
-contains
Description: Containment operator. Tells whether a single test value appears
in a set of reference values. Returns TRUE only when the test value exactly
matches at least one of the reference values. Contains uses reference equality
and returns a Boolean value.
Syntax:
<Reference-values> -contains <Test-value>
Examples:
C:\PS> "abc", "def" -contains "def"
C:\PS> "Windows", "PowerShell" -contains "Shell"
False #Not an exact match
# Does the list of computers in $domainServers
# include $thisComputer?
# -------------------------------------------
C:\PS> $domainServers -contains $thisComputer
-notcontains
Description: Containment operator. Tells whether a single (scalar) test
value appears in a set of reference values. Returns TRUE when the test
value is not an exact match for any of the reference values. Always
returns a Boolean value.
<Reference-values> -notcontains <Test-value>
C:\PS> "Windows", "PowerShell" -notcontains "Shell"
True #Not an exact match
# Get cmdlet parameters, but exclude common parameters
function get-parms ($cmdlet)
$common = "Verbose", "Debug", "WarningAction", "WarningVariable", `
"ErrorAction", "ErrorVariable", "OutVariable", "OutBuffer"
$allparms = (get-command $cmdlet).parametersets | foreach {$_.parameters} | `
foreach {$_.name} | sort-object | get-unique
$allparms | where {$common -notcontains $_ }
# Find unapproved verbs in the functions in my module
C:\PS> $approvedVerbs = get-verb | foreach {$_.verb}
C:\PS> $myVerbs = get-command -module MyModule | foreach {$_.verb}
C:\PS> $myVerbs | where {$approvedVerbs -notcontains $_}
ForEach
Sort
Tee
Where
-replace
Description: Replace operator. Changes the specified elements of a value.
C:\PS> "Get-Process" -replace "Get", "Stop"
Stop-Process
# Change all .GIF file name extension to .JPG
C:\PS> dir *.gif | foreach {$_ -replace ".gif", ".jpg"}
Equality Operators
The equality operators (-eq, -ne) return a value of TRUE or the matches
when one or more of the input values is identical to the specified
pattern. The entire pattern must match an entire value.
The following examples show the effect of the equal to operator:
C:PS> 1,2,3 -eq 2
C:PS> "PowerShell" -eq "Shell"
C:PS> "Windows", "PowerShell" -eq "Shell"
C:PS>
C:\PS> "abc", "def", "123" -eq "def"
Containment Operators
The containment operators (-contains and -notcontains) are similar to the
equality operators. However, the containment operators always return a
Boolean value, even when the input is a collection.
Also, unlike the equality operators, the containment operators return a
value as soon as they detect the first match. The equality operators
evaluate all input and then return all the matches in the collection.
The following examples show the effect of the -contains operator:
C:PS> 1,2,3 -contains 2
C:PS> "PowerShell" -contains "Shell"
C:PS> "Windows", "PowerShell" -contains "Shell"
C:\PS> "abc", "def", "123" -contains "def"
C:\PS> "true", "blue", "six" -contains "true"
The following example shows how the containment operators differ from the
equal to operator. The containment operators return a value of TRUE on the
first match.
C:\PS> 1,2,3,4,5,4,3,2,1 -eq 2
C:\PS> 1,2,3,4,5,4,3,2,1 -contains 2
In a very large collection, the -contains operator returns results
quicker than the equal to operator.
Match Operators
The match operators (-match and -notmatch) find elements that match or
do not match a specified pattern using regular expressions.
The syntax is:
<string[]> -match <regular-expression>
<string[]> -notmatch <regular-expression>
The following examples show some uses of the -match operator:
C:\PS> "Windows", "PowerShell" -match ".shell"
PowerShell
C:\PS> (get-command get-member -syntax) -match "-view"
C:\PS> (get-command get-member -syntax) -notmatch "-path"
C:\PS> (get-content servers.txt) -match "^Server\d\d"
Server01
Server02
The match operators search only in strings. They cannot search in arrays
of integers or other objects.
The -match and -notmatch operators populate the $Matches automatic
variable when the input (the left-side argument) to the operator
is a single scalar object. When the input is scalar, the -match and
-notmatch operators return a Boolean value and set the value of the
$Matches automatic variable to the matched components of the argument.
If the input is a collection, the -match and -notmatch operators return
the matching members of that collection, but the operator does not
populate the $Matches variable.
For example, the following command submits a collection of strings to
the -match operator. The -match operator returns the items in the collection
that match. It does not populate the $Matches automatic variable.
C:\PS> "Sunday", "Monday", "Tuesday" -match "sun"
Sunday
C:\PS>
In contrast, the following command submits a single string to the
-match operator. The -match operator returns a Boolean value and
populates the $Matches automatic variable.
0 Sun
The -notmatch operator populates the $Matches automatic variable when
the input is scalar and the result is False, that it, when it detects
a match.
C:\PS> "Sunday" -notmatch "rain"
C:\PS> "Sunday" -notmatch "day"
0 day
Replace Operator
The -replace operator replaces all or part of a value with the specified
value using regular expressions. You can use the -replace operator for
many administrative tasks, such as renaming files. For example, the
following command changes the file name extensions of all .gif files
to .jpg:
Get-ChildItem | Rename-Item -NewName { $_ -replace '.gif$','.jpg$' }
The syntax of the -replace operator is as follows, where the <original>
placeholder represents the characters to be replaced, and the
<substitute> placeholder represents the characters that will replace
them:
<input> <operator> <original>, <substitute>
By default, the -replace operator is case-insensitive. To make it case
sensitive, use -creplace. To make it explicitly case-insensitive, use
-ireplace. Consider the following examples:
C:\PS> "book" -replace "B", "C"
Cook
C:\PS> "book" -ireplace "B", "C"
C:\PS> "book" -creplace "B", "C"
book
Bitwise Operators
Windows PowerShell supports the standard bitwise operators, including
bitwise-AND (-bAnd), the inclusive and exclusive bitwise-OR operators
(-bOr and -bXor), and bitwise-NOT (-bNot). Beginning in Windows
PowerShell 2.0, all bitwise operators work with 64-bit integers.
Windows PowerShell supports the following bitwise operators.
Operator Description Example
-------- ---------------------- -------------------
-bAnd Bitwise AND C:\PS> 10 -band 3
-bOr Bitwise OR (inclusive) C:\PS> 10 -bor 3
-bXor Bitwise OR (exclusive) C:\PS> 10 -bxor 3
-bNot Bitwise NOT C:\PS> -bNot 10
-11
Bitwise operators act on the binary format of a value. For example, the
bit structure for the number 10 is 00001010 (based on 1 byte), and the
bit structure for the number 3 is 00000011. When you use a bitwise
operator to compare 10 to 3, the individual bits in each byte are
compared.
In a bitwise AND operation, the resulting bit is set to 1 only when both
input bits are 1.
1010 (10)
0011 ( 3)
-------------- bAND
0010 ( 2)
In a bitwise OR (inclusive) operation, the resulting bit is set to 1
when either or both input bits are 1. The resulting bit is set to 0 only
when both input bits are set to 0.
-------------- bOR (inclusive)
1011 (11)
In a bitwise OR (exclusive) operation, the resulting bit is set to 1 only
when one input bit is 1.
-------------- bXOR (exclusive)
1001 ( 9)
The bitwise NOT operator is a unary operator that produces the binary
complement of the value. A bit of 1 is set to 0 and a bit of 0 is set
to 1.
For example, the binary complement of 0 is -1, the maximum unsigned
integer (0xffffffff), and the binary complement of -1 is 0.
C:\PS> -bNOT 10
0000 0000 0000 1010 (10)
------------------------- bNOT
1111 1111 1111 0101 (-11, xfffffff5)
This came from doing this command but one of the fields was not exporting…
[PS] C:\scripts>Get-MailboxDatabaseCopyStatus e2k10DB01 | Select @{ n = "OutstandingDumpsterRequests";e={$_.OutstandingDumpsterRequests}} |export-csv test.txt
Use this for any Fail Exports to CSV or similar to this...."Microsoft.Exchange.Management.SystemConfigurationTasks.DumpsterRequestEntry[]"
You need to 1) use a Select Statement and 2) modify the Values not exporting.
Example.
Get-MailboxDatabaseCopyStatus e2k10DB01 | FL "OutstandingDumpsterRequests" | Export-CSV C:\Test.txt
This will be my only entry:
"Microsoft.Exchange.Management.SystemConfigurationTasks.DumpsterRequestEntry[]"
Note: I remove the Export-CSV and this shows the correct {} Value on the Screen.
So Next just to show this one Value.
Get-MailboxDatabaseCopyStatus e2k10DB01 | Select @{ n = "OutstandingDumpsterRequests";e={$_.OutstandingDumpsterRequests}} |Export-CSV C:\test.txt
My Entry will match the Output on the Screen
So why does this work? To specify a calculated property we need to create a hash table; that’s what the @{} syntax does for us. Inside the curly braces we specify the two elements of our hashtable: the property Name (in this case, OutStandingDumpsterRequest) and the property Expression (that is, the script block we’re going to use to calculate the property value). The Name property is easy enough to specify; we simply assign a string value to the Name, like so
n = "OutstandingDumpsterRequests"
And, believe it or not, the Expression property (which is separated from the name by a semicolon)isn’t much harder to configure; the only difference is that Expression gets assigned a script block rather than a string value:
e={$_.OutstandingDumpsterRequests}
Thus this outputs the Data into the CSV
@{ n = "OutstandingDumpsterRequests";e={$_.OutstandingDumpsterRequests}}
Does NOT export the data for Outstanding DumsterRequest….
[PS] C:\scripts>Get-MailboxDatabaseCopyStatus e2k10DB01 | Select "Identity","Name","DatabaseName","Status","MailboxServer","ActiveDatabaseCopy","ActivationSuspended","ActionInitiator","ErrorMessage","ErrorEventId","ExtendedErrorInfo","SuspendComment","SinglePageRestore","ContentIndexState","ContentIndexErrorMessage","CopyQueueLength","ReplayQueueLength","LatestAvailableLogTime","LastCopyNotificationedLogTime","LastCopiedLogTime","LastInspectedLogTime","LastReplayedLogTime","LastLogGenerated","LastLogCopyNotified","LastLogCopied","LastLogInspected","LastLogReplayed","LogsReplayedSinceInstanceStart","LogsCopiedSinceInstanceStart","LatestFullBackupTime","LatestIncrementalBackupTime","LatestDifferentialBackupTime","LatestCopyBackupTime","SnapshotBackup","SnapshotLatestFullBackup","SnapshotLatestIncrementalBackup","SnapshotLatestDifferentialBackup","SnapshotLatestCopyBackup","LogReplayQueueIncreasing","LogCopyQueueIncreasing","OutstandingDumpsterRequests","OutgoingConnections","IncomingLogCopyingNetwork","SeedingNetwork","ActiveCopy"| Export-CSV test.txt
But this DOES
[PS] C:\scripts>Get-MailboxDatabaseCopyStatus e2k10DB01 | Select "Identity","Name","DatabaseName","Status","MailboxServer","ActiveDatabaseCopy","ActivationSuspended","ActionInitiator","ErrorMessage","ErrorEventId","ExtendedErrorInfo","SuspendComment","SinglePageRestore","ContentIndexState","ContentIndexErrorMessage","CopyQueueLength","ReplayQueueLength","LatestAvailableLogTime","LastCopyNotificationedLogTime","LastCopiedLogTime","LastInspectedLogTime","LastReplayedLogTime","LastLogGenerated","LastLogCopyNotified","LastLogCopied","LastLogInspected","LastLogReplayed","LogsReplayedSinceInstanceStart","LogsCopiedSinceInstanceStart","LatestFullBackupTime","LatestIncrementalBackupTime","LatestDifferentialBackupTime","LatestCopyBackupTime","SnapshotBackup","SnapshotLatestFullBackup","SnapshotLatestIncrementalBackup","SnapshotLatestDifferentialBackup","SnapshotLatestCopyBackup","LogReplayQueueIncreasing","LogCopyQueueIncreasing",@{ n = "OutstandingDumpsterRequests";e=$_.OutstandingDumpterRequests}},"OutgoingConnections","IncomingLogCopyingNetwork","SeedingNetwork","ActiveCopy" | Export-CSV test.txt
Note: You may have to do this to several properties but it should work!
Why do we need another blog? Because our team has some pretty unique experiences we would like to share with the world, so hopefully you can use our knowledge to make your lives better, at least as it realtes to messaging! This blog will contain the collected knowledge of the Messaging Dedicated Support Engineers (DSE) for Microsoft's communications companies. In the messaging team, we work primarily with all versions of Exchange, Exchange ActiveSync (EAS) which includes non-Windows Mobile devices like Droids and iDevices, Blackberries, Enterprise Voice/Lync, Outlook, and integration of these technologies with other Microsoft products, such as Threat Management Gateway/User Access Gateway (TMG/UAG), Forefront Identity Manager (FIM), Active Directory, and others that seem to pop up in our day to day work. Thus, you can expect to read about a wide variety of issues and recommendations about a lot of technologies that interact with Exchange but not always just the back end mail infrastructure.
So, what is a DSE? DSE engineers work as a part of the Premier Field Engineer (PFE) family, a group of specialized engineers work directly with customers. PFE has two teams; transactional and dedicated. Transactional members tend to go from one customer to another as needed, providing some excellent onsite support and guidance to them. DSE engineers, on the other hand, typically have one to four customers who they work directly with on a (you guessed it) dedicated role. We work every day with some of Microsoft's largest customers who run our products. There are DSEs for nearly every Microsoft technology, so we are a big family. We like to focus on proactive work to prevent problems from occurring, such as operational improvements like backups and monitoring, but we are also there for our customers when technology or processes does not always work right.
What can you expect from this blog? Well, inside our Communications team, we have several members who deal with some of the largest communications companies in the world, so we see a lot of issues, technology and otherwise, that we would like to share. Now, issues do not always mean a fire drill or someone messed something up, on the contrary, I use the term to indicate something that could use improvement. So, you could expect to read a blog post about how to improve backups, tweaking authentication settings, undocumented features in PowerShell, and some really cool scripts for report generation. Part of the benefit of the communications industry is that it is very dynamic with the mergers and divestitures, so we asee large company issues as well as what can occur in smaller organizations.
What we will not do is divulge any confidential information about our customers, so please, don’t ask! We will work in personal anecdotes, stories, humor, tragedy, and anything else to make it interesting so it isn’t just your standard stale tech note. You can expect stories about cars, military history, pets, kids, spouses, and living in various regions of the States as well as travel around the world. Yet somehow, we will relate it back to messaging, promise!
We will try and update this blog at least once a month, possibly more depending on what information we want to/can share. We may even invite guest speakers from other portions of Microsoft to weigh in on discussions, such as AD team members or people from the product group.
Thanks again!