The pitfalls of LINQ deferred execution

Let’s face it, we all love the simplicity of Linq. The fluent syntax, the easy to read – almost sql-like – syntax. However, there are som pitfalls that I’ve seen colleagues fall into unknowingly. One of them is what is called  the deferred execution.

By design, you don’t execute a Linq command, you only specify it. The execution is not performed until the result is required. Hence deferred.

 

Take a look at the following code

Albeit a bit contrieved it is not an unusual pattern. I have a large list that i narrow down to a subset that I would like to work on (zip codes, gender) and then I examine this subset by looping through it.

On my machine this took 15000 ms (I’ve removed the Stopwatch stuff for clarity). This is not reasonable even though we have 1,000,000 records.

 

The reason is that listSmall is not a list (yet)! It is just a defined query. So, every time we execute

listSmall.Where(o => o == 100100 + i).Single()

we are, in fact, executing

list.Where(i => i > 100000 && i < 150000).Where(o => o == 100100 + i).Single()

So, instead of going through 50,000 records a 1000 times, we are searching 1,000,000 records! Not what we intended indeed. The way to solve this is to force Linq to execute the initial filter. The easiest way to do this is to simply append ToList() at the end. Like so:

var listSmall = list.Where(i => i > 100000 && i < 150000).ToList();

Now, the code runs in 400 ms. That’s what I call improvement.

 

Another scenario that has it cause in the same Linq feature. Inspect the following code

Here we have a recordset that contains a lot of users and I want to select and group all users from specific years. I loop through the set and select the users. I then store the result in an array. Imagine my surprise when I later loop through my yearbook and see that all users are from the same year. What happended?

Well, since I didn’t, actually, retreive the users in the first loop but only specified the query, when I finally did execute the query the loop is done and i == 0. Through closure, i is visible to my query snippet and is used to select users but – by then – it is 10.

 

The solution is once again to force execution by appending ToList() to the where statement at row 15.

Happy LINQing.

Debugging your iOS application with Fiddler

Many times I find myself in a situation where I need to monitor the http traffic from my apps. Normally – no problems – just go to Network settings -> Ethernet -> Advanced -> Proxy -> Check the http proxy and enter the address and port number of your fiddler instance.

Oh, and of course, your fiddler instance has to allow remote clients like so:

fiddlersettings2


However, when you want to connect through https, things are getting problematic. At least in the simulator. For the real device? No problemo. Just set the same setting for the proxy as above, and then browse (in Safari) to ipv4.fiddler:8888. This will open a web page generated from fiddler with a link to the certificate needed to allow Fiddler to act as man in the middle. Tell iOS to trust and install the certificate and you are good to go.

So. Now. The simulator has no network settings since it is using your Mac as a gateway. This means that when you’ve set your Mac to use Fiddler as a proxy, your simulator will too.

But when your code use the network stack, let’s say through NSURL, an exception will occur. The error will be something like

ERROR Error Domain=NSURLErrorDomain Code=-1202 “The certificate for this server is invalid. You might be connecting to a server that is pretending to be “xxx.azure-mobile.net” which could put your confidential information at risk.” UserInfo=0x7526530 {NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “xxx.azure-mobile.net” which could put your confidential information at risk., com.Microsoft.WindowsAzureMobileServices.ErrorRequestKey=<NSMutableURLRequest…

A nice, man-in-the-middle warning. DON’T GO THERE! However, since we actually want this scenario let’s make the simulator trust the certificate. The information about this is stored in a sqlite database in your ~/Library.

So, head over to ~/Library/Application Support/iPhone Simulator/<version>/Library/Keychains.

The you’ll find the TrustStore.sqlite3 database. If you’ve installed SQLiteManager, just doubleclick on the file. You’ll see a table (the only one) called tsettings. Before iOS 5 you just needed the SHA1 (of your certificate) as the lookup key and you were good to go. Not so anymore. Lucky for us there are plenty of Python scripts that will do this for us. Head over to Github and download iosCertTrustManager.py. Put it in /usr/local/bin.

Done? Good. Before we can use the script we have to get our hands on the actual certificate. Open, on your Mac (make sure it’s still in proxy mode), http://ipv4.fiddler:8888  and click on the certificate link and install it on your Mac. Now, open your KeyChain and find the certificate, right-click and export it as a PEM-file. Remember where you saved it.

Open a shell.

Execute

“chmod +x /usr/local/bin/iosCertTrustManager.py”

to allow execution. Now simple execute

iosCertTrustManager.py -a <path-to-the-certificate>/DO_NOT_TRUST_FiddlerRoot.pem”

assuming you kept the default name of the certificate. It will now start to ask questions like “.. import to v5.0?, v6.0?” and so on. Just answer yes to all of them.

Done….