F#: Downloading a file from behind a proxy
I’ve been continuing working on a little script to parse Cruise build data and the latest task was to work out how to download my Google Graph API created image onto the local disk.
I’m using the WebClient class to do this and the code looks like this:
let DownloadGraph (fileLocation:string) (uri:System.Uri) = async {
let webClient = new WebClient()
webClient.DownloadFileAsync(uri, fileLocation)}
Sadly this doesn’t work when I run it from the client site where I have access to the build metrics as there is a corporate proxy sitting in the way.
I tried Googling how to do this but all the ways that I tried kept resulting in the following error:
407 proxy authentication required
Even though I was entering a user name and password!
I didn’t succeed until my colleague showed me a way of getting past the proxy in C# which I could quite easily use in my code:
let DownloadGraph (fileLocation:string) (uri:System.Uri) = async {
let webClient = new WebClient()
webClient.Proxy <- new WebProxy("proxyName:port", true, null, new NetworkCredential("userName", "password", "corporateDomain"))
webClient.DownloadFileAsync(uri, fileLocation)}
One thing I was doing wrong was putting a 'http' at the start of the proxyName which I think in my case was wrong as I later learn that the proxy isn’t a HTTP one.
I’m also making use of asynchronous workflows in this example so that the actual downloading of the files will be done away from the main thread - this also gives me the option to download multiple files asynchronously if I want to.
About the author
I'm currently working on short form content at ClickHouse. I publish short 5 minute videos showing how to solve data problems on YouTube @LearnDataWithMark. I previously worked on graph analytics at Neo4j, where I also co-authored the O'Reilly Graph Algorithms Book with Amy Hodler.