Opening checkout

VPS and infrastructure12 min read

Fix RDP Error 0x3 0x11 on Windows Server 2022

Fix RDP error 0x3 0x11 on Windows Server 2022: turn off the WDDM display driver, UDP on your PC and extra monitors first. Every other fix, step by step.

By QuantVPS Team

Published
Updated
Reviewed by
Todd, Trading Analyst

To fix RDP error 0x3 0x11 on Windows Server 2022, turn off the WDDM graphics display driver for Remote Desktop on the server, turn off UDP on your own PC and lower the monitor settings. The error usually appears after the session connects, so a closed port is rarely the cause. It is a bigger problem on a trading VPS, where you need reliable access to your trading platform.

The Remote Desktop client shows "Error code: 0x3" with "Extended error code: 0x11". Microsoft's list of disconnect codes gives code 3 as "Your Remote Desktop Services session has ended, possibly for one of the following reasons: The administrator has ended the session. An error occurred while the connection was being established. A network problem occurred." The session usually connects and then drops, so the display driver, UDP and monitor fixes below are the first things to try.

Here’s how you can resolve the error:

  • Display and UDP Fixes First: Disable the WDDM graphics display driver policy on the server, turn off UDP on your own PC and lower the monitor settings (details in the first fix below).
  • Check RDP Services: Ensure Remote Desktop Services are running. Use services.msc or PowerShell to verify and restart if necessary.
  • Firewall Rules: Confirm TCP port 3389 is open and properly configured in Windows Defender Firewall.
  • User Permissions: Verify that users are added to the Remote Desktop Users group and aren’t blocked in Local Security Policy.
  • Registry Settings: Check keys like fDenyTSConnections and SecurityLayer to ensure they’re correctly configured.
  • Network Level Authentication (NLA): Temporarily disable NLA to test compatibility and re-enable it after resolving the issue.
  • Advanced Fixes: Address role-specific RDP issues, optimize system performance, and resolve licensing or configuration errors.

For traders, these steps ensure uninterrupted access to trading platforms, minimizing downtime and risks. Strengthen long-term stability by enabling features like NLA, updating certificates, and monitoring logs for potential security threats. If the session drops after idle time instead, see how to keep your RDP session alive. A QuantVPS Windows Server VPS can host your trading platform.

Keep your platform running while you're away.

Setting Up Your System for Troubleshooting

Before tackling the fixes for RDP Error 0x3 0x11, it's essential to prepare your system properly. A well-configured setup not only saves time but also ensures your troubleshooting efforts are directed at the root cause. Keep in mind that Windows Server 2022 Essentials requires manual RDP configuration. Once the basics are in place, reviewing your system and network settings will help pinpoint the source of the issue.

System and Network Requirements

Getting your system and network settings right is key to resolving RDP Error 0x3 0x11.

Port Configuration and Network Access
Start by confirming that TCP port 3389 is open. You can do this by running the command:
netstat -ano | find ":3389".

Next, ensure Windows Defender Firewall allows traffic through TCP 3389. Open Windows Defender Firewall with Advanced Security and verify that the necessary inbound and outbound rules for Remote Desktop connections are active.

User Access Configuration
Make sure remote connections are enabled by navigating to the System Properties. Add the required users to the Remote Desktop Users group to grant appropriate access.

How to Fix RDP Error 0x3 0x11 Step by Step

Now that your system is configured correctly, let’s examine resolving RDP Error 0x3 0x11. Follow these steps in order, starting with the most common fixes. After each step, test your RDP connection to see if the error is resolved.

Start With the Display Driver, UDP and Monitor Settings

Error 0x3 with extended code 0x11 usually appears after the session has connected, so a closed port is rarely the cause. Try these three fixes first, and test the connection after each one.

  1. Turn off the WDDM display driver for RDP (on the server). Run gpedit.msc and go to Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Remote Session Environment. Set Use WDDM graphics display driver for Remote Desktop Connections to Disabled, then restart Windows. Remote Desktop then uses the older XDDM display driver.
  2. Turn off UDP on the client (on your own PC). In gpedit.msc, go to Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Connection Client and set Turn Off UDP On Client to Enabled, so RDP traffic uses TCP only. The policy is stored as the DWORD value fClientDisableUDP set to 1 under HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\Client.
  3. Lower the display and monitor settings. In Remote Desktop Connection, click Show Options, open the Display tab, untick Use all my monitors for the remote session, and pick a lower resolution and color depth. If the error stops, add monitors back one at a time.

If none of these helps, work through the checks below in order.

Check RDP Service Status

First, ensure that the Remote Desktop Services are running. Open the Services console by pressing Windows + R, typing services.msc, and hitting Enter. Locate Remote Desktop Services in the list and check if its status is "Running." If it’s stopped, right-click on it, select "Start", and set the startup type to "Automatic" to avoid future interruptions.

For a more detailed check, open PowerShell with administrator privileges. Run the command Get-Service -Name "TermService" to verify the service status. If it’s not running, start it using Start-Service -Name "TermService". You can also check related services like Remote Desktop Configuration and Remote Desktop Services UserMode Port Redirector using the same commands.

Additionally, check the Terminal Services registry key. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server and confirm that the fDenyTSConnections value is set to 0. If it’s set to 1, RDP connections are disabled, and you’ll need to change it to 0.

Fix Firewall and Network Problems

Once you’ve verified the RDP service, address any potential network or firewall issues. Open Windows Defender Firewall with Advanced Security and check both inbound and outbound rules for Remote Desktop. Look for rules allowing traffic on TCP port 3389. If these rules are missing or disabled, create new ones or enable the existing ones.

To create a new inbound rule:

  • Open the Actions panel and choose "New Rule."
  • Select "Port" as the rule type, choose TCP, and specify port 3389.
  • Allow the connection and apply the rule to all profiles (Domain, Private, and Public).
  • Name the rule something clear, like "RDP Allow Inbound."

Also, review Network Level Authentication (NLA) settings in System Properties under the Remote tab. While NLA adds security, it can sometimes cause compatibility issues with certain client setups. Temporarily disable NLA to see if it resolves the error. If it does, re-enable it once the connection is stable.

After addressing network issues, ensure user permissions are configured correctly.

Review User Access and Security Settings

If the network setup is correct, the issue could be related to user access. Open Local Users and Groups by running lusrmgr.msc. Under Groups, double-click Remote Desktop Users and confirm that the necessary users are added to this group.

Next, check Local Security Policy settings for any restrictions. Run secpol.msc and navigate to Local Policies > User Rights Assignment. Look for Allow log on through Remote Desktop Services and ensure your user or group is listed. Also, check Deny log on through Remote Desktop Services to confirm your account isn’t explicitly blocked.

If issues persist, review Group Policy settings for Remote Desktop Services. Run gpedit.msc and go to Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Connections. Make sure policies like Allow users to connect remotely by using Remote Desktop Services are set to "Enabled" or "Not Configured."

Finally, inspect the registry key at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp. Check the SecurityLayer value. A value of 2 enforces SSL/TLS encryption, which may cause compatibility problems with older RDP clients. You can temporarily set it to 1 (Negotiate) or 0 (the older RDP Security Layer, which uses native RDP encryption without TLS server authentication) to test the connection. Set it back to 2 as soon as the test is done, because the lower settings weaken security on an internet-facing trading server.

Password policies can also block RDP access. If you’re using a local account, ensure the password meets Windows complexity requirements. Note that RDP doesn’t allow connections with empty passwords, even if local logons do.

Advanced Fixes for Difficult Cases

When basic troubleshooting doesn't resolve your Remote Desktop Protocol (RDP) issues, it's time to examine advanced configurations. These solutions tackle more intricate problems, such as misconfigured settings or performance bottlenecks, ensuring your trading operations run smoothly.

Remote desktop

Your trading computer. In the cloud.

Connect and control your trading strategies from any device, anywhere in the world, through secure remote desktop, while execution remains on QuantVPS.

Choose your cloud trading computer
One session on every screen: your phone, iPad, Mac and Windows PC all open the same live desktop on your VPS.

Fix Remote Desktop Role Problems

If your server uses multiple Remote Desktop Services roles - like Remote Desktop Session Host, Connection Broker, or Gateway - it's essential to confirm that each role is configured correctly. Start by checking the Remote Desktop Services deployment overview in Server Manager for any errors or warnings.

For issues with the Session Host role, verify that your licensing is set up correctly. Misconfigured or expired licenses can disrupt connections, so use Windows Server's management tools to confirm everything is in order.

If you're facing problems with the Connection Broker, ensure it has the necessary access to Active Directory. You might need to adjust service account permissions to fix access issues.

Additionally, double-check that RDP port 3389 is open and properly configured. You can also run the sfc /scannow command to repair any corrupted system files that might be affecting your connection.

Improve RDP Performance for Trading

Once you've resolved role-specific issues, it's time to enhance overall system performance. Optimized performance is key to reducing latency and ensuring reliable trading executions.

  • Resource Allocation: Make sure your server is using all available CPU cores. For instance, if you're on a VPS Pro plan from QuantVPS with 6 cores, open System Configuration (msconfig.exe), go to Boot > Advanced options and leave Number of processors unchecked, so Windows uses every core.
  • Memory Management: Adjust performance settings through Advanced System Settings. You might also benefit from setting processor scheduling to prioritize background services, which can improve RDP session stability.
  • Network Optimization: Fine-tune your network settings to support stable RDP connections. Be sure to consult official Windows Server documentation before making changes to ensure they align with your security requirements.
  • Multi-Monitor Support: If you're using multiple screens, confirm that your Remote Desktop client is configured to support multi-monitor setups.
  • Audio Redirection: Disable audio redirection if you don't need it. This small adjustment can help prevent unnecessary connection interruptions.
  • Session Timeout Settings: Modify session timeout settings through the Remote Desktop Session Host configuration or Group Policy. Proper timeout settings can help keep your session active during brief network interruptions without risking disconnections.
  • GPU Acceleration: If your server includes dedicated graphics hardware, enabling GPU acceleration can reduce CPU usage and improve responsiveness, especially for trading platforms that rely on graphics-intensive tasks.

Maintain Stable and Secure RDP Connections

Once you've tackled RDP Error 0x3 0x11, it's time to focus on keeping your connections stable and secure. For trading operations, where uninterrupted access and security are critical, maintaining your Windows Server 2022 environment is an ongoing priority.

Strengthen Security Measures

Start by enabling Network Level Authentication (NLA). This feature requires users to authenticate before initiating a full RDP session, effectively stopping many automated attacks. You can activate NLA through the System Properties dialog under the Remote tab.

Next, enforce strong password policies. Passwords should be at least 12 characters long and include a mix of uppercase and lowercase letters, numbers, and special characters. Combine this with account lockout policies to defend against brute-force attacks.

To secure your RDP sessions further, use certificate-based authentication. Configure your Windows Server 2022 to use SSL certificates, ensuring all data between your local machine and the server is encrypted. This is especially important when accessing your trading VPS from public or unsecured networks.

Regular security audits are a must. Keep an eye on Windows Event Logs to monitor both successful and failed logon attempts. Unusual patterns here could indicate potential breaches or compromised accounts.

Finally, restrict RDP access by implementing IP whitelisting, allowing only trusted IP addresses to connect. This reduces your system's exposure to external threats.

Conclusion: Main Points to Remember

Fixing RDP Error 0x3 0x11 on Windows Server 2022 calls for a step-by-step approach that tackles various possible causes. Start by checking the basics: service status, firewall rules, user permissions, and advanced configurations. This structured troubleshooting process is essential for resolving the error and avoiding disruptions to trading operations.

Be cautious with Windows updates - some cumulative updates have been known to interfere with RDP functionality. If necessary, roll back problematic updates to restore access.

If the issue persists, the Microsoft Remote Display Adapter could be the culprit. Removing it via Device Manager often resolves stubborn RDP problems.

Beyond troubleshooting, strengthening your RDP security can stabilize connections over the long term. Use measures like Network Level Authentication, certificate-based authentication, and IP whitelisting to safeguard your trading environment while ensuring more dependable RDP access. Regularly reviewing Windows Event Logs can also help you catch and address potential problems before they lead to connection failures.

RDP Error 0x3 0x11 often stems from conflicts with display adapters, network problems, or Group Policy settings. Keep a detailed record of your troubleshooting steps to streamline future fixes and maintain smooth trading operations. A clear and organized approach is key to keeping your systems running securely and efficiently.

Order flow

Built for scalpers. Heatmaps on dedicated resources.

Order-flow heatmaps and DOM ladders run on dedicated CPU, memory and NVMe, so your charts keep responding during volatility.

See the dedicated hardware
Order flow through a news release: resting bids and offers paint in as a heatmap beside the DOM ladder, then a burst of buys takes the offer wall. Illustrative; prices are decorative.

If RDP error 0x3 0x11 comes back

If the error returns after a Windows update, check the update history first; rolling back the update that broke RDP is often faster than repeating every fix. Keep a note of the working settings so you can restore them quickly. QuantVPS Windows Server VPS plans list each server's RDP credentials in the dashboard's control panel and include always-on DDoS protection with advanced filtering.

FAQs

What causes RDP Error 0x3 0x11 on Windows Server 2022, and how can I troubleshoot it?

RDP Error 0x3 0x11 on Windows Server 2022 usually stems from network connectivity problems, Group Policy conflicts, display adapter issues, or incorrect Network Interface Card (NIC) configurations. Here’s how you can tackle it:

  • Check your network connections: Ensure there are no disruptions or unstable connections that could interfere with Remote Desktop.
  • Review Group Policy settings: Use gpedit.msc to look for any conflicting policies that might be causing the error.
  • Inspect display drivers: Confirm your display adapter is functioning properly. If necessary, update or reinstall the drivers.
  • Verify NIC configuration: Make sure the NIC is set up correctly, either as a private or domain network, depending on your environment.

Focusing on these areas often resolves the issue and helps restore a stable Remote Desktop connection.

How can I set up my Windows Server 2022 for reliable and secure RDP connections, especially for trading?

To maintain stable and secure RDP connections on your Windows Server 2022 - especially when used for trading - start by bolstering your security measures. Changing the default RDP port is a simple yet effective way to lower the chances of automated attacks. Additionally, configure your firewall to permit RDP access exclusively from trusted IP addresses for an added layer of protection.

Another critical step is to enable multi-factor authentication (MFA) and ensure all passwords are strong and unique. This significantly reduces the risk of unauthorized access.

Regular updates are essential - keep your server and all installed software patched to address potential vulnerabilities. Monitor your server’s performance regularly and limit user access based on specific roles to reduce exposure to unnecessary risks. These practices help establish a secure and dependable setup designed for trading activities.

What should I do if basic fixes don’t resolve RDP Error 0x3 0x11 on Windows Server 2022?

If basic troubleshooting doesn’t fix RDP Error 0x3 0x11 on Windows Server 2022, here are some advanced steps to try:

  • Review Group Policy settings: Open gpedit.msc and check for any misconfigurations related to the Remote Display Adapter. Incorrect policies can disrupt RDP functionality.
  • Update or reinstall display drivers: Outdated or faulty display drivers are often behind this issue. Make sure to update them or reinstall as needed.
  • Inspect network settings: Check your NIC detection, domain membership, and overall network configuration. Connectivity problems are a frequent trigger for RDP errors.

These steps target system-level problems and can help restore reliable remote desktop access.

Disclaimer: QuantVPS does not represent, guarantee, support, or endorse any third-party brands, products, or services mentioned in this article. All brand references are for informational purposes only. This information does not constitute a recommendation to trade futures or any other financial instruments. All trading decisions are made at your own discretion. Please be aware that futures trading involves significant risk of loss, and past performance does not guarantee future results. Read our full Brand Non-Endorsement Disclaimer.

Risk Disclosure: QuantVPS does not provide financial, investment, or trading advice. Trading involves substantial risk of loss and is not suitable for every investor. Past performance is not indicative of future results. You should consult a qualified financial advisor before making any trading decisions. Read our full Trading Disclaimer.