Pictures from Ghent

My love of old cities had already taken me down to Rothenburg ob der Tauber and Heidelberg, so I don’t know why I had never been to Ghent, which is a lot closer. However, while all of these places feature beautiful medieval architecture, Ghent feels the most vibrant. Unlike Rothenburg, which forbids any changes that alter the cities appearance, Ghent feels more alive as it also incorporates modern elements in beautiful symbioses. And it is build around several rivers and canals which – as you can probably tell from the number of photos – has its very own appeal.

Day 1

The weather the first day wasn’t great, so all of that day’s pictures a bit gloomy.




Day 2

I got a very early start on the second day and was rewarded by these beautiful sights during the blue hour.




Mond über Kanal

Alte Post

On my way home, a final look back at the church towers that define Ghent’s skyline and are visible from almost anywhere in the city.

Look back


The Case of Multicast Message Loss (again)

I have written about trouble-shooting multicast issues several times before, but multicast is a gift that keeps on giving.

The Problem

The application in question would miss a substantial number of messages. A trace on the connected switch showed that all packets had been put on the wire. Tracing with Microsoft Message Analyzer on the machine showed these same messages missing, so our application probably was not at fault. Additionally, it did work on other machines just fine.

The Analysis

So I went back to the drawing board, reviewed and double-checked everything I had learned about high-throughput multicast messaging and

  • set appropriately large socket receive buffer sizes in the multicast message receiving application,
  • activated all TCP/UDP Rx/Tx offloads in the NIC configuration,
  • activated receive side scaling (RSS) and picked the maximum number of RSS queues,
  • set the NICs’ receive buffers to their maximum values,
  • disabled flow-control,
  • turned off all power-saving features in NIC and operating system,
  • used the most aggressive interrupt moderation setting, and
  • updated the NIC driver top the latest version.

In order to check NIC settings, I keep the following PowerShell snippet handy. It gives me the current, all valid and maximum values for each parameter of each NIC in the NIC team. And it doesn’t even require admin privileges.

(Get-NetLbfoTeam "MyNicTeam").Members | Get-NetAdapterAdvancedProperty | ft DisplayName,DisplayValue,ValidDisplayValues,NumericParameterMaxValue

Other useful sources:

But even after ensuring all parameter were at their optimal values, the problems persisted. So I spent some time setting up perfmon with these network-related performance counters.

One counter immediately jumped out: Packets Received Discarded was pretty much constant on the machines our application worked on. But on the machines where we noticed packet loss, this number was growing fast.

This Technet blog post has a good explanation of that performance counter and tips on how to gather it from multiple machines remotely using PowerShell.

The Cause

It turns out the machines experiencing multicast message loss had substantially smaller receive buffers (512) compared to the machines that were working fine (2048 and 4096). Even though our setup script had correctly configured the maximum value for this parameter, that was apparently still insufficient.

So we ended up upgrading the NICs on the cluster experiencing the problems and the multicast messages loss went away.

Upon closer examination we also noticed TCP packet loss while our multicast application was running. But because resends were mostly successful, only introducing small delay this had gone unnoticed before.

Pictures from Lübeck Part 1: The Sights

Over the Easter holiday I spent two days in Lübeck, Northern Germany. With some of the highlights already published on my Instagram, here’s the full report.

Coming from the train station, you enter the city through Holstentor, its most iconic landmark.

Holstetor, St. Petri, Salzspeicher

Holstentor SPQL

Lübeck Cathedral with Mühlenteich.

Dom und Mühlenteich

The townhall below is particularly curious: it combines several different materials, colors and styles while also lacking a clear architectural focal point.

Rathaus (1)


In general, I just fell in love with the historic crow-stepped gable town houses you find everywhere.

Häuser (2)

Häuser (1)

Häuser mit Halbturm

Model of the historic city center with Holstentor on the left, town hall in the center and the cathedral at the bottom: