This document actually started as a combination of several threads from the email reflector for the Internet Gateway Device (IGD) Working Committee for the UPnP™ Forum (http://www.upnp.org) and newsgroups like microsoft.public.upnp. There were several questions regarding implementing and troubleshooting IGDs based upon UPnP™ Device Control Protocols (DCPs) (http://www.upnp.org/standardizeddcps/igd.asp) that were not functioning as expected. As part of the process, we tried to define how we thought the UPnP™-based Control Point (CP) User Interface (UI) shell extensions in Microsoft’s Windows XP Operating System work, how they “interact” with the IGD, what they display, and what the protocols on the wire look like when communicating with the IGD.
The real issues turn out to be that
It is not documented;
Things appear to work differently based on different versions of the OS and UPnP™-based applications that use the services;
It works differently with Windows’ Internet Connection Sharing (ICS) as the IGD than with most “hardware-based” IGDs;
There are different behaviors exhibited depending upon the optional UPnP™ services actually implemented in the IGD device;
The “proper” results are dependent upon certain services running on the client computer. These services are not enabled or running by default.
Fortunately, there was a lot of input from those actually working on developing IGDs and who were able to make sense out of most of it. This document is an attempt to define how we understand the UPnP™-based communications with an IGD to work with Windows XP SP-1, although some of the changes we encountered when using a non-SP-1 release are documented.
It assumes that the reader has at least a working understanding of UPnP™ communications (information regarding the UPnP™ Device Architecture is available from the UPnP™ Forum’s web site at http://upnp.org).
It also assumes the reader is familiar with the functionality provided by IGDs and Network Address Translation (NAT) Traversal. A white paper covering the Windows XP UPnP™ architecture from a developer’s view is available here. Additional information on Microsoft’s UPnP™ Client Support is available here.
Note again that Microsoft has not sanctioned this document and is in no way connected to it. Although the UPnP™ Forum also does not endorse it, we are grateful to many of its members for the additional information they have supplied.
We do not delve into all aspects of the various permutations; just those we think are useful for troubleshooting and relevant to a better understanding of the functionality, as we know it. As with most “standards” and implementations in this industry, they seem to get enhanced or modified constantly, so this information is subject to change as new updates to the OS and associated applications and services are available. Consider this the simplified version of how things appear to interoperate (and actually do for several of the products currently on the market). We also have tried to detail the known exceptions.
An IGD is found using the UPnP™ Discovery process;
An icon is displayed in the System Notification Area (what used to be termed the System Tray located next to the clock) indicating that a new UPnP™ device has been discovered;
An icon appears in the Network Connections (NC) folder identified as an Internet Gateway Device. Depending upon what services are implemented in the device, this icon allows one to bring up the presentationPageURL and control the IGD and NAT functionality through the Windows XP customized CP UI;
An icon sometimes appears in My Network Places (MNP) depending upon what services are implemented in the device and, like the one in NC, allows one to control the IGD through what initially appears to be the same CP UI;
One runs an application like Windows Messenger or a game written to use Microsoft’s DirectPlay (DPLAY) Application Programmer Interfaces (APIs) and “NATted” ports are opened and closed dynamically on the IGD to support communications across the Internet.
It turns out that there is a lot more to it than that. A general remark up-front: Our first mistake was to assume the Control Point for NC and MNP is the same. It turns out that NC and MNP use independent Control Points, which do not relate to each other, each with its own discovery mechanisms.
There also is a third more obscure “Control Point” of interest built into DirectPlay, as part of DirectX 8.1 or later. DirectPlay (when tested with Windows XP SP-1) uses its own UPnP™-based IGD discovery and control processes, which are not the same as those for NC.
It is used, for example, by Windows Messenger for XP Version 4.7. On the other hand, it turns out that MSN Messenger 5.0 (which is different than Windows Messenger) does use the NC discovery and control mechanisms, as detailed later.
Following are the UPnP™-based communications aspects as defined via the collaborative efforts of the group as of this writing. They are broken into sections based upon the different ways in which Windows XP appears to interact with different IGD device attributes.
WINDOWS XP’S UPnP™ SERVICES
A. IGDs and NAT Traversal
UPnP™ support for Internet gateways has become important since Windows XP implemented NAT Traversal (based on the IGD model). Many Windows applications will not work correctly through a gateway unless it acts as an IGD. Actually, many users see the terms “UPnP™” and “IGD” more or less as synonyms where Internet Gateways are concerned.
NAT Traversal effectively means that the functionality of an Application Level Gateway (ALG) moves from the gateway itself to the Windows PC. The gateway no longer needs built-in knowledge about which port mappings to set up or how to handle the data streams for a particular application. Instead, the application tells the gateway which port mappings it needs (through UPnP™ functionality) and transparently adjusts the data streams accordingly (depending upon the gateway’s external IP address, also obtained through the UPnP™ interfaces available at the time).
Windows XP not only contains Control Points for interacting with IGDs, it also contains its own UPnP™-based device implementation through Internet Connection Sharing (ICS). ICS is a NAT-enabled Internet Gateway Device, which supports the UPnP™-based IGD device model. In many cases, ICS makes a good starting point for studying the behavior expected by a “minimalistic” IGD. [The CP and the ICS IGD were developed together and pretty much define how they interoperate. It turns out that the ICS IGD was the only IGD available at the time the CP code was developed, so it was the only one the CP was tested with at the time.]
B. Associated UPnP™ Services
Windows XP includes several Operating System-level services for supporting UPnP™ technologies. These UPnP™-oriented services are:
Simple Services Discovery Protocol (SSDP) service – Facilitates the discovery of UPnP™-based devices on the network. This service must be running or the machine will not see the IGD (or other UPnP™-based devices). In fact, this service must be running for the UPnP™ non-host APIs to function. Other Control Points like those for DirectPlay do not need it;
UPnP™ Device Host – Provides support for hosting UPnP™-based devices within the machine;
Application Layer Gateway Service – Provides support for 3rd party protocol plug-ins for Internet Connection Sharing and the Internet (not required for UPnP™) functionality, but it is related).
C. Windows Components
Some parts of UPnP™ support in Windows XP become enabled by adding certain Windows components (Control Panel -> Add or Remove Software -> Add/Remove Windows Components -> Networking Services). In particular, if something does not work as expected, check to see that you have installed each of the required Windows components listed above.
In Windows XP pre-SP-1, there is only one component – “Universal Plug and Play,” which enables UPnP™ support in My Network Places. UPnP™ support in Network Connections works independently of the installation of this component. In other words, this optional component only affects My Network Places. The other APIs and services always are installed in Windows XP pre-SP-1.
In Windows XP SP-1, however, there are two components:
“Universal Plug and Play” – Enables UPnP™ support in My Network Places;
“Internet Gateway Device Discovery and Control Client” – Enables UPnP™ support in Network Connections.
These two components do not depend on each other.
UPnP™ support in Windows Messenger 4.7 currently works independently of the installation of either of the two Windows components above. UPnP™ support in Windows Messenger 5.0, though, requires the “Internet Gateway Device Discovery and Control Client” to be enabled. Rumor has it that, as of this writing, Messenger 6 leverages UPnP™ technologies a little differently.
IV. MY NETWORK PLACES
It is important to note that My Network Places (MNP) is a generalized CP for most UPnP™ devices with presentation pages – it is not restricted to IGDs. This CP also exists in Windows ME.
My Network Places discovers all UPnP™ root devices and retrieves their Device Description Documents (DDDs). The DDDs must be valid (in particular, they must contain at least one service). They also must contain a presentationPageURL, or MNP ignores them. When both criteria are met and the device goes “alive,” an icon representing the root device is displayed in MNPs. Double-clicking this icon opens the device’s presentation page in the user’s Internet browser.
The icon in MNPs should disappear either
When it broadcasts a “byebye”; or
When the timeout on the “alive” advertisement expires.
Notice that we do not use the term “shortcut” to describe the icon. It really is a specialized Control Point UI “icon” built into Network Places and is not a typical Windows “shortcut” as we normally think of one.
If there is no presentationPageURL passed back in the Device Description Document (it is optional, after all), My Network Places simply does not show this UPnP™ device. An example is Windows XP’s current ICS implementation, which is an IGD without a presentationPageURL defined. Note that the presence or non-presence of a presentationPageURL does not affect the visibility of an IGD in Network Connections (see below).
The UPnP™ Forum may drop the check for having at least one service present in the root device some day (the UPnP™ Forum is looking into this in conjunction with the recently ratified BasicDevice DCPs); after all, MNP does not use any of the device’s services, only the DDD and the presentationPageURL. Currently, though, there is no way around this.
Windows XP SP-1 seems to have introduced stricter checks for the Device Description. In particular, the XML directive now must be present at the beginning of the DDD file (this was not required in Windows XP pre-SP-1).
In contrast, Windows ME uses an M-SEARCH message, which really does not conform to the UPnP™ standard, as the quotes around “ssdp:discover” are missing in the MAN header. If you want to support Windows ME, you have to respond to this variant as well.
So far, Windows XP ignores any tags in the Device Description Document. Instead, MNP uses its own icons for the UPnP™ devices it displays. Usually, a standard icon is shown (a computer screen on a LAN segment, for instance). IGDs, however, seem to get a different icon – a flat screen with a globe in the background; a camera icon is seen as well when working with UPnP™-enabled cameras (in addition to others).
The icons come from %windir%system32upnpui.dll. [If you want to see them, create some dummy shortcut on the desktop, click “Change icon …” in the Properties dialog, and navigate to upnpui.dll].
The question of how MNP picks an icon still needs further investigation. More specifically, it currently is unknown which element in the Device Description Document (if that really is what it is checking) is used by the decision logic when displaying a specific icon image for the device in addition to the descriptor ranges used for selecting each of the icon images.
C. Additional Observations
A UPnP™-based “New device found” icon is displayed in the System Notification Area (next to the clock) whenever a new UPnP™ device goes “alive” on the network. It currently does not appear consistently based upon who is logged on to the machine at the time the discovery is made. This icon will linger
1. Until you click on it;
2. Until the device sends out a “byebye” message;
3. Until the user logs out and back in again; or
4. Until the next re-boot – after which the device no longer is considered “new.”
MNP uses the standard IE cache for the Device Description files. When a UPnP™ device goes down and comes up again, MNP does not retrieve the Device Description Document from the device again. It uses the cached copy instead. This may cause problems if the Device Description has changed in between since MNP wouldn’t notice (even if the UDN of the root device has changed). The cache is cleared by rebooting the Windows XP system, however.
V. NETWORK CONNECTIONS
The following applies to the Control Point (CP) in Network Connections (NC) only
- The Control Point in Network Connections does an Async search for urn:schemas-upnp-org:device:InternetGatewayDevice:1 when the system boots. [Similarly, MNP does a search for all root devices. See below for how Windows Messenger handles it];
- Windows XP listens for UPnP™ “alive” advertisements. Note that the NC CP drops all known devices and does a new search when it detects a change in IP address. As far as we can tell, the MNP Control Point does not do this;
- If a UPnP™-based device is discovered and there are no malformed HTTP headers, malformed XML schemas, premature “byebye” messages, or timeouts, it looks for a deviceType of urn:schemas-upnp-org:device:InternetGatewayDevice:1;
- It does an HTTP:GET and downloads the Device Description Document (DDD) from the IGD if Windows XP sees the appropriate deviceType. As of Windows XP SP-1, the IGD also must be the default gateway for the XP system (it must appear in the routing table as the gateway for destination 0.0.0.0, netmask 0.0.0.0). Otherwise, the DDD simply gets downloaded, but nothing else happens;
- It looks for the WANCommonInterfaceConfig and WANPPP/IPConnnection services in the DDD at this point. It uses the serviceType (urn:schemas-upnp-org:service:WANPPPConnection:1 or WANIPConnection:1), and not the serviceId [it turns out that these specific serviceIds were not specified by the UPnP™ Forum when the CP was written];
- For each WAN connection, it invokes GetConnectionTypeInfo and GetNATRSIPStatus. The connection must have the following properties
- ConnectionType must not be Unconfigured (and it should be IP_Routed);
- NATEnabled must be 1 (or True). If there are multiple WAN connections in the Device Description Document, NC loops over them until it finds one with the right ConnectionType and NATEnabled. It stops once it has detected the first appropriate connection and does not display more than one;
- The specialized Windows XP NC Control Point UPnP™ shell UI then shows an icon in Network Connections (NC);
Otherwise, the NC CP simply ignores the connection and the icon does not show up.
B. Port Mappings
When listing the IGD’s port mappings for each PortMappingIndex, Network Connections first invokes GetGenericPortMappingEntry and then GetSpecificPortMappingEntry. NC loops until GetGenericPortMappingEntry returns an error (usually SpecifiedArrayIndexInvalid). PortMappingNumberOfEntries is ignored.
Although NC retrieves all port mappings, it only displays entries with
PortMappingDescription != “”;
RemoteHost == “”;
LeaseDuration == 0.
Similarly, all port mappings added through NC have these properties.
If PortMappingDescription ends in ” [MICROSOFT]” (there is a space just before the opening bracket), NC treats the port mapping in a special way. The user may not delete the entry and may change only PortMappingEnabled and InternalClient. This probably is implemented this way to protect ICS’s built-in port mappings. Note that although the CP retrieves the full description from the IGD (Microsoft’s ICS for example), NC’s CP UI displays it without the trailing ” [MICROSOFT].”
C. Additional Observations
Windows XP SP-1 only talks to and allows one to control an IGD that also is the default gateway. This characteristic is a result of a security update Microsoft implemented in Windows XP SP-1. When they were looking at potential threats to the system, they decided that somebody could pull off a spoof against IGDs, so they decided one way to limit the threat is to talk only to IGDs that are the default gateway. As an aside, if your IGD also is NOT your default gateway, then all DPLAY (the Direct Play component of DirectX) applications that use DPLAY to traverse NATs (like many games or Windows Messenger) break as they only do unicast discovery requests (as opposed to multicast) and subsequently control ONLY the default gateway.
Microsoft’s ICS supports the following (Microsoft-specific) extensions to the IGD model
The OSInfo:1 service (urn:schemas-microsoft-com:service:OSInfo:1) with the OSMajorVersion, OSMinorVersion, OSBuildNumber, and OSMachineName variables;
The X_PersonalFirewallEnabled and X_Uptime variables in addition to the X_GetICSStatistics action in WANCommonInterfaceConfig;
The X_Name variable in WANIPConnection.
The exact service descriptions can be found in the %SystemRoot%system32icsxml folder on your Windows XP system.
When an IGD supports these extensions, there are the following differences
NC shows the connection as “X_Name on OSMachineName” instead of the usual “Internet Connection;”
NC uses X_GetICSStatistics (single action) to retrieve statistics instead of GetTotalBytesSent and others (six actions).
These extensions are not required for an IGD to be recognized properly and handled by NC, however.
VI. MSN MESSENGER VERSION 5.0
MSN Messenger 5.0 uses the same NC Discovery functionality discussed above to find the IGD. Note that this is in contrast to the process outlined below for Windows Messenger 4.7 (these are different executables).
VII. WINDOWS MESSENGER FOR XP VERSION 4.7
Windows Messenger 4.7 uses DirectPlay for NAT Traversal, so everything said here also applies to DirectPlay in general (as of DirectX 8.1). We are concentrating on Windows Messenger here, though, since it is by far the most important DirectPlay application for IGD developers to understand.
When Windows Messenger Version 4.7 starts, it sends several M-SEARCH requests for WANIPConnection:1 and WANPPPConnection:1. These requests are sent to the first default gateway of the Windows XP system – and they are sent as unicast requests, not multicasts. The “first” default gateway is considered the one that has the lowest metric. Windows Messenger will not try any other gateways that may exist.
When the gateway responds, Windows Messenger retrieves the Device Description Document (DDD) and uses the first WANIPConnection:1 or WANPPPConnection:1 service appearing in the DDD to add its port mappings. It does not check ConnectionType or NATEnabled, as mentioned under Network Connections (V.A.6 above). It also does not check ConnectionStatus or ExternalIPAddress. Note that this behavior is quite different from NC’s and may lead to Windows Messenger’s using another gateway (or connection) than the one shown by NC.
Also note that there is a potential inconsistency if the IGD supports more than one connection (say, one WANPPPConnection and one WANIPConnection). If only one of them is configured at a time, NC always shows the right one. Windows Messenger Version 4.7, however, effectively requires that a device developer constructs the DDD so that the configured connection appears first. Otherwise, Windows Messenger will use the wrong connection.
We have tried to convey what we discovered so far concerning how Windows XP SP-1 interacts with UPnP™-based Internet Gateway Devices. As mentioned, some of this is subject to clarification and may change as additional patches and updates are released. We hope that it contains useful information to help IGD developers and implementers understand and troubleshoot the process.
Well-deserved thanks go to Thomas Betker from Siemens. His input, insight, and proof reading have been invaluable in the creation of this document. Thanks again Thomas.
Please send all comments, suggestions, updates, and further observations for potential inclusion in an updated version to:
Derek R. Flickinger is Vice President of Research and Development for Interactive Homes, Inc., a consulting, design, implementation, and development company focused on delivering standards-based Distributed Audio, Video, Communications, and Control (DAVCC) services in the home. His long-term goal is to be instrumental in the development and deployment of DAVCC-based services on space stations.
UPnP™ is a certification mark of the UPnP™ Implementers Corporation.
MicrosoftÂ® and WindowsÂ® are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of other companies and products mentioned herein may be the trademarks of their respective owners.