Museum Quadrat Bottrop

For this year’s museum visit on Christmas (cf. #Weihnachtsmuseumsbesuch) the family went to the museum Quadrat in Bottrop. The exhibition itself wasn’t particularly exciting, but the museum architecture and its location in a large park more then made up for that.

Quadrat was also unusual for housing both natural history, a permanent exhibition on the history of the city, and contemporary art in the same museum.


A few Hours in Lisbon

Having four days to discover Porto (see part 1 and part 2), I figured I should also take the train down to Lisbon, and thus cross off another item from my list of European Capitals to travel to.

This, however, was a mistake. Not that Lisbon isn’t worth a visit. On the contrary. I should have allocated more time to it. Especially since the train ride was a good three hours in either direction and days in November are already getting shorter.

While I took a bus tour around the city center, I ended up spending most of my time in Parque das Nações which has a beautiful beach front featuring exciting modern architecture.

Pictures from Porto: Part 1

To escape the cold, rainy gray of November in Germany, I spent a few days in Porto which had perfect weather with mostly clear skies and temperatures of 15-20°C.

Below is the Livraria Lello bookstore which has such a beautiful interior, they charge 5€ to enter. And it’s totally worth it.

I think the only other time that I was able to eat ice-cream under palm trees on my birthday was four years ago in Hong Kong.

Windows Server Multicast Networking Guide

This article is a compilation of several articles on the topic of networking (particularly multicast messaging) for Windows Server that I had previously written. Many of the hits on this blog recently have been for these articles, so I figured it was time to compile all my learnings from several years of working with high-throughput applications on Windows in one convenient place.

Sub-par Performance because of Power-Saving Features

Both Windows as well as network interface cards (NICs) come with various power saving features. The default settings are usually a compromise between energy efficiency and performance. For optimal performance they will need to be changed.

This article shows how to do that at the operating system level: Power-Saving in Windows Server 2008 R2 and later

Not Receiving Multicast Messages on a Microsoft Failover Cluster

It can happen that multicast messages are not received on a machine that is part of a Failover Cluster because the physical interface has a higher metric than the virtual failover adapter that is installed as part of Microsoft Failover Cluster.

This article explains how to diagnose the issue and adjust that metric of the physical interface so applications listening for incoming multicast messages are joined on the correct interface: The Case of Multicast Messages not being Received on a Windows Server 2008 R2 Microsoft Failover Cluster

Multicast Messages dropped because of a small Socket Receive Buffer

As messages travel from the wire through the NIC, the operating system network stack to your application, there are various buffers along the way that will need to be large enough to hold data for a little while in case one of the components involved is busy.

As an application developer, it is your job to make sure the receive buffer sizes used by the sockets listening for incoming multicast messages are large enough to buffer data e.g. when a garbage collection prevents your code from processing it for a moment.

Multicast Messages dropped because of the Base Filtering Engine

If there is a lot of multicast traffic on a machine, Windows’ Base Filtering Engine (BFE) might not be able to keep up with it, resulting in dramatic message loss. BFE is part of Windows Firewall, but it might be active even when a different firewall solution is used.

This article links to a hotfix available for Windows Server 2012 and also explains how to fix the issue in Windows Server 2012 R2 through the UdpExemptPortRange registry parameter: The Case of Multicast Message Loss on Windows Server 2012 R2

Multicast Messages dropped because of a small NIC Receive Buffer

Another reason for multicast message loss I experienced has been too small of a buffer on the receiving NIC. Everything was going smoothly until traffic was becoming a bit spiky. So even though the application socket receive buffer (see above) was large enough, data didn’t reach it because it was dropped in the network stack.

This article shows how to review NIC parameters with PowerShell (which unlike the NIC driver GUI in Windows does not require administrator access) and diagnose this kind of message drops: The Case of Multicast Message Loss on Windows Server 2012 R2 (again) It also contains some general tips on optimal NIC settings.

Multicast Message dropped because of the wrong Receive Side Scaling Load Balancing Profile

This article takes a look at the RSS Load Balancing Profile and how a driver update resetting it caused multicast message loss: The Case of Multicast Message Loss because of the wrong Receive Side Scaling Load Balancing Profile

The Case of Multicast Message Loss because of the wrong Receive Side Scaling Load Balancing Profile

[2018-11-11 Update] I have compiled information from this and all my other articles on the topic into my Windows Server Multicast Networking Guide.

The Problem

Even after I had applied all the optimizations outlined in my Windows Server Networking Guide, I was still experiencing multicast message loss and even noticeable delays in TCP traffic on one Windows Sever 2012 R2 machine in a failover cluster.

Interestingly enough, this was an issue on only one of the cluster nodes but not others. My analysis showed, it was a machine which hosted applications that are essentially using one and only one CPU continuously for mathematical calculations.

The Cause

During installation of the NIC and a subsequent driver update, one of the NIC parameter was apparently reset: the Receive Side Scaling (RSS) Load Balancing Profile.

While Microsoft’s documentation lists the default as NUMAScalingStatic, in my case it was set to ClosestProcessor (the exact parameter and value names may vary based on your NIC model).

If I understand the documentation and Microsoft’s Introduction to Receive Side Scaling correctly, ClosestProcessor will essentially use just one processor for handling all network traffic. This would line up neatly with my observation that only one machine with heavy-single-CPU-use applications was affected.

The Solution

Adding a line to the machine setup PowerShell script to force the parameter back to NUMAScalingStatic has completely resolved the issue.