Fiddler: Trying to work out how it all hooks together
I mentioned previously that we’re making use of Fiddler quite a lot on my current project, mainly to check the traffic going to and from the service layer, and I’m quite curious how it actually works.
In particular I wanted to know:
-
How we’re able to route requests through Fiddler and then through the corporate proxy
-
How proxy settings work differently for Firefox and Internet Explorer
As far as I’m aware the source code for Fiddler isn’t available so a colleague and I tracked the various proxy settings when Fiddler was turned on and off and also had a look at some registry settings.
As Internet Explorer is a WinInet based application it looks at the Local Area Network settings to check whether there are any proxies that it needs to route requests through. I believe any proxy defined in here would be known as the 'system proxy'.
We noticed that our application also routed its requests via the proxy defined in the LAN settings as well - it is a .NET application but I guess that even applications written in other languages but running on windows would make use of the system proxy too?
Firefox has its own proxy settings which are accessible via the preferences menu of the browser and are then configurable from the 'Advanced -> Network' tab.
If another proxy is defined in the LAN settings when Fiddler starts up it takes this value and stores it somewhere (in memory presumably) as the 'Upstream gateway proxy' (Tools -> Fiddler Options -> Connections) which it directs requests through before they go out to the internet.
It then changes the proxy defined in the LAN settings to point to itself - typically port 8888 locally.
The 'Upstream gateway proxy' always seems to be the same as the 'system proxy' so if a different proxy is defined on FireFox than Internet Explorer this won’t be used even though on the latest version of Fiddler when it starts up the FireFox proxy will be changed to point to Fiddler.
When Fiddler is shut down it reverts the proxy settings to how they were previously although (unsurprisingly) if it crashes then we are left in a state where requests are being sent to 'http://localhost:8888' and there is no application listening on that port anymore meaning that we end up with a 'Cannot find server or DNS Error'.
Restarting Fiddler gets rid of that problem although the 'Upstream gateway proxy' usually seems to get lost so if we want to make requests out to the internet then we need to go and reset that in our LAN settings before restarting Fiddler.
If there’s anything inaccurate in what I’ve written or if you know of any resources that would help me understand this better if you let me know in the comments that’d be cool!
About the author
I'm currently working on short form content at ClickHouse. I publish short 5 minute videos showing how to solve data problems on YouTube @LearnDataWithMark. I previously worked on graph analytics at Neo4j, where I also co-authored the O'Reilly Graph Algorithms Book with Amy Hodler.