F#, TryWith, Maybe and Umbrella

By Jay at December 06, 2008 13:49 Tags: , , ,
Cet article est disponible en Français.
 
I've thrown myself a bit in the discovery of F#, and even though I do not intend to make it my first language, I intend to use techniques and features found in it and try to port them into C#. New additions in C# 3.0 make it a good target for functional concepts.

There seem to be a consensus for the fact that F# is not a multi-purpose language, as C# is also not, for instance with the writing of parallel code. C# is not a perfect language for this, but F# seems to be. At the opposite, F# does not seem to be a language of choice for writing GUI code. For my part, and considering that F# if not really official, reusing concepts will be enough for now.

TryWith Extension

Using F#, I had to write this:


   let AllValidAssemblies = [
        for file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll") ->
        System.Reflection.Assembly.LoadFile(file)
    ]

This code creates a list of assemblies that can be loaded in the current AppDomain. There is however an issue with the invocation of the Assembly.LoadFile method, because it raises an exception when the file is not loadable for some reason. This is a non-modifiable behavior, even though we would like to return a null instead of an exception.

To work around this, there is a feature in F# that can do this :


    let EnumAllTypes = [
        for file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll") ->
        try System.Reflection.Assembly.LoadFile(file) with _ -> null
    ]

The point of the try/with block is to transform any exception into a null reference.

To transpose the creation of this list in C# with a LINQ query, the same problem arises. We must intercept the exception raised by LoadFile and convert it to a null reference.

Here is the equivalent in C#, without the exception handling :


    var q = from file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll")
            let asm = Assembly.LoadFile(file)
            select asm;

When LoadFile raises an exception, the execution of the request is interrupted, which is a problem.

Extension Methods can be of a great value here, and even though a normal method could do the trick, we can write this :


    public static class Extensions
    {
        public static TResult TryWith<TInstance, TResult>(this TInstance instance, Func<TInstance, TResult> action)
        {
           try {
              return action(instance);
           }
           catch {
              return default(TResult);
           }
        }
    }


The idea behind this method is to reproduce the behavior of the try/with F# construct. With this method, we can update the LINQ query into this :


    var q = from file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll")
            let asm = file.TryWith(f => Assembly.LoadFile(f))
            select asm;


This is creates the same list the F# code does, with null references for assemblies that could not be loaded.

The TryWith method can be overloaded to be a bit more flexible, like calling a method for a specific exception :



    public static TResult TryWith<TInstance, TResult, TException>(
           this TInstance instance, Func<TInstance, TResult> action,
           Func<TException, TResult> exceptionHandler
        )
        where TException : Exception
    {
        try {
           return action(instance);
        }
        catch (TException e) {
           return exceptionHandler(e);
        }
        catch {
           return default(TResult);
        }
    }


By the way, there is an interesting bug with this code. If we execute this :


    string value = null;
    var res = value.TryWith(s => s.ToString(), (Exception e) => "Exception");


The behavior is different depending on whether it is executed with or without the debugger with the x86 runtime. It seems that the code generator "forgets" to add the handler for the TException typed exception, which is annoying. This is not a big bug, mainly because it only appears when the x86 debugger is present. With the x64 runtime debugger, there is no problem though. For those interesting in this, the bug is on Connect.

Maybe Extension

I also added recently in Umbrella an extension named Maybe, which has a behavior rather similar to TryWith, but without the exceptions :


    public static TResult Maybe<TResult, TInstance>(
         this TInstance instance,
         Func<TInstance, TResult> function)
    {
       return instance == null ? default(TResult) : function(instance);
    }


The point of this method is to be able to execute code if the original value is not null. For instance :


    object instance = null;
    Console.WriteLine("{0}", instance.Maybe(o => o.GetType());


This allows the evaluation of the GetType call, only if "instance" is not null. With a method call like this, it is possible to write an "if" block, with inside a LINQ query, this because a bit more complex.

The idea for the code is not new and is similar to the functional Monad concept. It has been covered numerous times, and an implementation more in line with F# can be found on Matthew Podwysocki's Blog.

Pollution ?

When we're talking about pollution linked to Extension Methods, we're talking about Intellisense pollution. We can quickly find ourselves dealing with a bunch of extensions that are useful in the context of the current code, which renders Intellisense unusable. With Umbrella, these two extensions are somehow polluting all types, because they are generic without constraints.

Although these are only two very generic extensions, this can apply to almost any block of code, but they could find themselves better placed in an Umbrella Extension Point, rather than directly on every types.

We could have this :



    object instance = null;
    Console.WriteLine("{0}", instance.Control().Maybe(o => o.GetType());


But I have some issues with this approach : To be able to create an extension point the Control methd has to create an instance of the IExtensionPoint. This add a new instantiation during the execution, although the lambda also creates itself a hidden instance, we're counting anymore... There is also the fact that it lengthens the code line, but is only aesthetics. We'll see what pops out ...

Anyway, it is interesting to see the impact the learning a new language has on the style of writing code with another language that one's been using for a long time...

Local Variables in Lambda Expressions

By Jay at November 20, 2008 19:58 Tags: ,
Cet article est disponible en français.
 
After a quick chat with Eric Lippert about a post  on the use in lambda expressions of a local variable declared in a foreach loop, Eric pointed me out that this piece of code :

    int a = 0;
    Action action = () => Console.WriteLine(a);
    action();

 
Is actually not expanded by the compiler to this code :

    [CompilerGenerated]
    private sealed class <>c__DisplayClass1
    {
       public int a;

       public void <Main>b__0()
       {
          Console.WriteLine(this.a);
       }
    }

    void Main()
    {
       int a = 0;

       var display = new <>c__DisplayClass1();
       display.a = a;

       var action = new Action(display.<Main>b__0);

       action();
    }

 
I made the assumption that a local variable was simply copied in the "DisplayClass" if it is not used after the creation of the lambda, which  is not the case.

If we take this slightly different sample :


    int a = 0;
    Action action = () => Console.WriteLine(a);
    a = 42;
    action();

 
My assumption would have made this code display "0". This is correct because lambda expressions (and anonymous methods) "capture" the variable and not the value; The execution must display 42.

Actually, this latter piece of code is expanded like this:

    var display = new <>c__DisplayClass1();

    display.a = 0;

    var action = new Action(display.<Main>b__0);

    display.a = 42;

    action();


 
We can see that in fact, the variable that was previously local, on the stack, has been "promoted" as a field memberr of the "Display Class". This means that all references to this "local" variable, inside or outside of the lambda, are replaced to point to the current instanc e of the "DisplayClass".

This is quite simple actually, but we can feel that the "magic" behind the C# 3.0 syntactic sugar is the result of a lot of thinking !

I will end this post by a big thanks to Eric Lippert, who took the time to answer me, even though he's probably under heavy load with the developement of C# 4.0. (With the contravariance of generics, yey !)

Lambda Expression and ForEach loops

By Jay at November 18, 2008 20:58 Tags: , ,

Cet article est disponible en français.

To enhance the performances of a type serializer, and to use a small extension that I recently wrote for Umbrella I stumbled upon an interesting small "Side Effect" seen when creating lambda expressions inside a foreach loop.

Let's take this simple piece of code :


    var actionList = new List<Func<int>>();

    foreach (var value in Enumerable.Range(0, 10))
    {
       actionList.Add(() => value);
    }

    actionList.ForEach(func => Console.Write("{0} ", func()));

Which outputs this :


   9 9 9 9 9 9 9 9 9 9



Which is, of course, what we could have expected.

Lambda expression have the ability to use variables that are in the scope when they are declared. This makes them very interesting, but to properly use them, it is best to understand how they are "materialized" by the compiler.

Like a lot of features of C#, like the using, foreach, iterators or lock, lambdas are syntactic sugar destined to simplify the writing of code that is most of the time pretty verbose. It is possible to write the expanded code for these keyswords in C#.

Let's take this other piece of code :


    int a = 0;
    Action action = () => Console.WriteLine(a);
    action();


The lambda expression is "materialized" by the C# compiler under the form of a "Display Class", that allows the storage of the local variable "a" :



    [CompilerGenerated]
    private sealed class <>c__DisplayClass1
    {
       public int a;

       public void <Main>b__0()
       {
          Console.WriteLine(this.a);
       }
    }

We can see that the indentifiers for the generated class are not valid in C#, but are valide from the CLR point of view. We can also see that the local variable used during the declaration is present as a member variable, in the class that contains the code of the lambda expression. The compiler will then write this to create an instance of the lambda expression :


    int a = 0;

    var display = new <>c__DisplayClass1();
    display.a = a;

    Action action = new Action(display.<Main>b__0);

    action();


There also, this is not valid C#.

But then, what happens for the foreach case so that the content of the variable is repeated ?

If we analyze the first code sample generated by the compiler with Reflector, there is nothing much fancy to see with the C# visualizer :


    List<Func<int>> actionList = new List<Func<int>>();
    using (IEnumerator<int> CS$5$0000 = Enumerable.Range(0, 10).GetEnumerator())
    {
       while (CS$5$0000.MoveNext())
       {
          int value = CS$5$0000.Current;
          actionList.Add(delegate {
             return value;
          });
       }
    }



The lambda expression is represented as an anonymous method, which is a synonym of lambda, but that does not explain the behavior.

We must look at the generated IL to understand the behavior, and this is the correct C# code that is generated :


    List<Func<int>> actionList = new List<Func<int>>();
    using (IEnumerator<int> CS$5$0000 = Enumerable.Range(0, 10).GetEnumerator())
    {
       var myLambda = new <>c__DisplayClass4();

       while (CS$5$0000.MoveNext())
       {
          int value = CS$5$0000.Current;

          myLambda.value = value;

          actionList.Add(new Func<int>(myLambda.b_0));
       }
    }



We can easily see what the problem is : The instance of the class containing the lambda is created only once, and reused many times to assign a new value for each iteration. This explains why the execution of all the lambdas return the last enumerated value, because they all refer to the same instance of the "DisplayClass" type.

However, if we write the code this way :


    foreach (var value in Enumerable.Range(0, 10))
    {
       int myValue = value;
       actionList.Add(() => myValue);
    }

The behavior changes, and this time, each lambda has the correct value.

From the compiler point of view, the creation of a new instance for the container class must be the consequence of the creation of a new variable. For the case of a ForEach statement, this is not the case and the variable is treated as created only once, then reused..

So, from the compiler point of view, a ForEach loop is expanded like this :


    using (var it = Enumerable.Range(0, 10).GetEnumerator())
    {
       int value;

       while (it.MoveNext())
       {
          value = it.Current;
          actionList2.Add(() => value);
       }
    }



This is probably a question of interpretation, but I wasn't exactly expecting this...

So, one must pay attention to the way local variables are used in lambda expressions, depending on their declaration location.

I'll explain in a later post why I did have to use lamda expressions in a ForEach loop.

Working with Umbrella in .NET 3.5

By Jay at November 08, 2008 16:06 Tags: ,
If you've been using .NET 3.5, and the new features that are provided by C# 3.0, and especially LINQ, you must have wondered why, oh why, there isn't an extension named ForEach on an IEnumerable<T>.

Well, I still haven't figured that out, though it appears that it might have something to do with the fact that by nature an Action<T> is most of the time not "pure", which means that it modifies some states, somewhere. I can't remember where I've found that explanation, though it "might" make sense from a functional point of view.

But you may wonder why, then, is there a ForEach method on List<T> ? Well, I don't know why there is that kind of consistency issue, but I know there is one library that somehow tries to fix this :
 
 
This library fills the gaps left by the BCL, and adds a whole bunch of new extension methods that eases the use of .NET 3.5 classes. This library is not a framework though, mainly because you don't need to redesign your whole application to use it.

A few weeks ago during a presentation of Umbrella at the .NET User Group of Montreal, the creators Francois Tanguay and Erik Renaud from nVentive, were asked the question "Where do we start ?". This is a tough question to answer, because of the nature of umbrella, which "plugs" itself everywhere it can.

Using Umbrella

Let’s see a simple example:

    Enumerable
        .Range(20, 30)
        .ForEach(
            (index, value) => "Value #{0}={1}".InvariantCultureFormat(index, value)
        );

As you can see, using Umbrella requires most of the time the use of lambda expressions. Here, this example is an answer to the question "How can I get the index of the enumerated value in a foreach statement ?".

Another extension provided by Umbrella is the Remove method on an ICollection by specifying a predicated to select elements to be removed.

            var values = new List<int>(Enumerable.Range(1, 10));
            values.Remove(v => v > 4);

Sure, you can write some code that does this, but you'll have to create a temporary list to store elements to be removed, then remove them from the collection.

There is also an extension on IDictionary<> that simplifies the use of TryGetValue which requires an “out” argument, which is particularly annoying to write. Umbrella provides a TryGetValueOrDefault that either gives you the value for the key you requested, or default(TValue) if the key is not found.
 

The Action.ToDisposable() extension method

One particularly interesting extension is the ability to encapsulate an Action delegate into an anonymous disposable instance. Let’s say that we need to profile the duration of a particular scope of code. We would need to write something like this :


    var w = Stopwatch.StartNew();
    for (int i = 0; i < 1000; i++)  { }
    Console.WriteLine(w.Elapsed);



We would have to write the enclosing code to time for each portion of code we would need to profile. You could of course write a class that would do this on purpose :


    class ProfileScope : IDisposable
    {
        Stopwatch w = Stopwatch.StartNew();
        void IDisposable.Dispose()
        {
            Console.WriteLine(w.Elapsed);
        }
    }


And it’s being used like with a “using” statement :


    using (new ProfileScope())
    {
        for (int i = 0; i < 1000; i++)
        {
        }
    }

This time, timing a block of code is somehow easier to write, however, there is a way to simplify the writing of the writing of the utility class by doing this:

    static IDisposable CreateProfileScope()
    {
        var w = Stopwatch.StartNew();
        return Actions.Create(() => Console.WriteLine(w.Elapsed)).ToDisposable();
    }

We can use here the ToDisposable extension to be able to call the specified action in the dispose method of some disposable class.

The point of this code is to avoid exposing the inner details of the profiling code, and just expose a known and generic way for executing something at the end of a scope.

Extension Points

There is one drawback when using extension methods: They tend to “pollute” the original type. By pollute I mean that they appear in the IntelliSense window. Because it is easy to add extension methods, you end up having hundreds of new methods on a type, which make IntelliSense much less useable.

The Umbrella guys came up with the idea of Extension Points, which are a way to group extension methods the namespaces do for types.

This is somehow a hack from a code perspective, but is rather ingenious from a usability perspective. For instance, extending the serialization applies to every type that exists, and placing all serialization extension methods directly on System.Object is not a good idea.

Extensions points are used like this:

    using (MemoryStream stream = new MemoryStream())
    {
        int a = 0;
        a.Serialization().Binary(stream);
        // ...
    }



Serialization is an extension method placed on every object that returns an SerializationExtensionPoint instance, and the “Binary” method extends that type instead of System.Object.

It is important to remember that Umbrella is for most of its code is not "rocket science", as it probably contains code that you may already have partially developed for your own project. It's only the amount of small but useful utility methods that Umbrella provides that makes it worth using.

I’ll be writing a few other posts about Umbrella; there are a lot of extensions in there that are great time savers.

 

Speaker Idol Montreal

By Jay at November 04, 2008 06:56 Tags: ,

Yesterday, Monday 3rd, was the day of Speaker Idol Montreal.

I'd like to thank every attendee for being here and listening all of these great presentations ! Ten minutes is a format that's a bit rough though, one's got to learn every word to be said to make a good point during that kind of presentation length. (And don't rely on the computer that's available there... You might end up not being able to use the keyboard :) )

Anyway, if you were interested by the subject of my presentation, you may find a more thorough explanation on this blog post about DataBinding in WPF. By the way, you may also be interested in this post about DataBinding Application Settings.

See you next time in the .NET Montreal User Group !

Montréal Bus Stop Locator Site and the iPhone

By Jay at November 01, 2008 11:13 Tags:

Cet article est disponible en francais ici.

I've updated my utility WebSite to look a bit better on the iPhone (Thanks, Mr Sanx).

Now, the next buses schedule are a bit easier to read and past schedules are visually easier to leave out.

There are some other features that are in the oven, I hope to push them online soon enough :)

The geolocation on the iPhone would be a plus, but unfortunately, Google Gears is not available on the iPhone. There seem to be some sort of extension named weblocate:// to do this from a web application, but that seems a bit shady...

Anyway, you can visit the Web Site here from your Windows Mobile, iPhone, or desktop computer at this address :

http://jaylee.org/stm

 

Here's an example of what it's looking now :

   

   

How to convert a (big) VMWare VMDK into an Hyper-V VHD

By Jay at October 16, 2008 19:36 Tags:

Cet article est disponible en francais ici.

I've been playing a lot with Hyper-V lately and quite frankly, I'm very pleased with it.

VMs a very responsive, the IO is not the bottleneck it was before, the impact of VMs on the host is far lower than with VPC or VMWare, it supports snapshots and fine grained ACLs on each VMs.The performance part is subjective as always, but I find it faster, and better integrated in the OS that other products. And now, there is a free version of Windows 2008 Server named Hyper-V Server, which allows to run a bare minimum text-mode only version of windows Just Hyper-V and the VMs.

By the way, I partially agree with Paul Thurrott on the "GUI user experience" of Hyper-V which is not very elegant, involving some scripting and information found on blogs, but hey, this is a server product. This is definitely not for the average joe that does not know a bit of what he is doing.

I've had recently to port a VMWare VM to Hyper-V and the main disk was created as a fixed length (flat) -- disk of 80GB, which is obviously a VMDK file of 80GB.

First, I tried converting the file with the VMDK to VHD converter, which unfortunately does not seem to support big flat disks. I already tried converting disks with this tool, I know for a fact that is does work, so it must be the size of the file.

Then I tried using the VMWare Virtual Disk Manager to convert the flat VMDK to a multiple 2GB VMDK spanned file. After the conversion, the VMDK to VHD converter worked perfectly by converting my spanned VMDK to a flat VHD disk compatible with Hyper-V.

This does not end here however, because mass storage drivers installed by Windows at install time for VMWare VMs are not compatible with the one Hyper-V is using. This leads to a nice BSOD saying INACCESSIBLE_BOOT_DEVICE, described here and by the KB314082.

There's two ways to fix this : Create the reg file from the KB article and merge it when the VM is running under VMWare, or mount the VHD disk into an other Hyper-V VM and merge the reg file in the SYSTEM hive of the target OS.

In my case the first possibility was out of the question; Moving the disk again to another machine would have been a waste of time.

That left me with the registry hive mounting solution. Here's how to integrate the registry file :

  • Use another VM to mount the target VHD, to be able to see the SYSTEM hive file.
  • Using the regedit, load the system hive from System32\config\system into HKLM\temp.
  • Modify the KB reg file replace every reference to "SYSTEM\CurrentControlSet" by "temp\ControlSet001", and import it to update the default boot configuration
  • Modify again the KB reg file to replace every reference to "SYSTEM\CurrentControlSet" by "temp\ControlSet002", to modify the "Last Known Good Configuration", and import it , just in case.
  • Unload the hive.
  • stop the current VM.
  • Boot the new VM using the converted VHD disk, and voilà !
This is time consuming, but worth the trouble, the VM is now working properly under Hyper-V.

Pocket IE and Setting IMG Src via JavaScript

By Jay at September 07, 2008 09:28 Tags:

Cet article est disponible en francais ici.

With the Montreal's Bus Stop Locator web site, to be able to use Google Maps effectively, it's imperative to let the client's browser fetch the image by itself. The Google API does not allow a single key to grab a lot of maps from a single host, so redirecting the image is not an option.

I wanted to have the image Url generated so I could use the Width of an HTML body to be embedded in the Map query, so I created a small script that builds the URL and sets as the SRC of the IMG element that will contain the Map.

Well, that seems easy, but it's not... The IMG does fetch the image, but the element's size is not updated with the size of the new image.

In my case, I'm placing the image in a TABLE element, so I'm getting the default cell size for my image. To fix that I also had to set the width and height of the IMG element, to get it to have a proper size.

PocketIE has little support for scripting, but it's somehow clear that it has been a hard to fit addition to the engine... Come on microsoft ! You can do better than that ! Have a look at what Opera's been doing...

Montreal Bus Stop Locator Site Updates

By Jay at September 07, 2008 09:06 Tags: ,

Cet article est disponible en français ici.

So far, the response has been great for my little utility, even though I'm not making an active advertising.

Using some Microsoft terminology, I'm dogfooding my own application and I found a few points that could be enhanced a bit :

  • Now, if using Google Gears, you are not automatically redirected to the stop list for your location. You simply see a small map of where you're supposed to be, and you're offered a other button to choose to use that location.
  • Some error messages are displayed when typing incorrect or missing data. That could be confusing...
  • All maps are now using all the available width of the screen. That means that the image is not resized poorly on a mobile device. Still, I wonder how the iPhone wil behave with this...
  • I fixed a bit the distance display to display Km instead of meters if appropriate, even though if you are in an other city, you'll still be offered some reaaaally long distance.
    I'm still wondering if I need to filter out people that are outside Montreal. The point of this site is also to be a technological proof-of-concept for GeoLocation, so even if the distance is not usable, it's still a valuable information.
  • I've also added some street name samples to help people that are not from Montreal to see what the site can do. I've taken a random intersection and bus stop code that can be typed in directly.
  • I also made some XHTML compliance, for what it's worth :)

Now, I think I'll try to make sure that Google search will find the site in both French and English languages because right now, I'm having the same problems I did have with my Remote Control software pages. Google's crawling without specifying a language, so for this site, English is going to come out, since it is the default language. Maybe I'll add a "/fr" and "/en" virtual base for each of them.

I'm always interested in any suggestions or bug reports !

.NET 2.0 InternalsVisibleTo Attribute and Unsigned Assemblies

By Jay at September 02, 2008 07:19 Tags:

Ce post est disponible en francais ici.

In a recent project, for the purpose of unit testing, I had to use the InternalsVisibleTo attribute, which extends the scope of the internal qualifier. This allows the separation of the unit testing code assembly from the actual code, without publishing the internals to the "public". This way, you can avoid shipping your unit testing code.

This is an interesting attribute from many points of view, but when using it, you may face this nice error message :

error CS1726: Friend assembly reference 'Dummy' is invalid. Strong-name signed assemblies must specify a public key in their InternalsVisibleTo declarations.

Problem was, my current assembly nor the target assemblies were signed. I tried adding a dummy PublicKey or PublicKeyToken as indirectly suggested here and here, but as many people out here, I don't want to mess with assembly signing at this point of my project.

It turns out that the compiler considers your assembly as "signed" if there is either an AssemblyKeyFile or AssemblyKeyName defined on the assembly, even though both of them are empty. 

So, to be able to use AssemblyKeyName InternalsVisibleTo with unsigned assemblies, just remove AssemblyKeyFile or AssemblyKeyName attributes if you don't use them.

About me

My name is Jerome Laban, I am a Software Architect, C# MVP and .NET enthustiast from Montréal, QC. You will find my blog on this site, where I'm adding my thoughts on current events, or the things I'm working on, such as the Remote Control for Windows Phone.