Two Wrongs

.NET on Non-Windows Platforms: A Brief History

.NET on Non-Windows Platforms: A Brief History

My current job consists to a large degree of writing C# code 1 Fortunately, also F# code to some degree. Unfortunately, if you are not a citizen of Microsoft land2 I.e. if you use Microsoft Windows, develop in Microsoft Visual Studio, host your websites on the Microsoft iis web server, use Microsoft Team Foundation Server for version control, cut your videos in Microsoft Movie Maker, and reach out to your family with Microsoft msn Messenger, – if you are using Linux and write your code in Emacs – Microsoft leaves you out in the cold, and getting started with .net is really confusing. Really, really confusing.

Despite how complex this is, and how bitter I may sound when writing about it, please keep these two things separate:

In order to reduce confusion for the next person, I’ll try to summarise what I’ve learned about the platform and how it got to the sorry state it is. I think this is easist to understand through the lens of history, so that’s where we’ll start.

First of all, I’m going to define the abbreviation nwp to mean “non-Windows platform” – I’m not sure yet how much I’ll come to use that phrase in this article, but I end up using it a lot when discussing this subject with others.

Note that this article will not contain any practical advice on how to get going. The situation is confusing enough that this long article is needed just to clear out the types of terminology used. Hopefully, armed with the knowledge you gain from this, you can be more successful in setting things up on your own.

Historic Context

Those who know their Brief, Incomplete, and Mostly Wrong History of Programming Languages know what happened between the mid ’90s and early ’00s:

1996 - James Gosling invents Java. Java is a relatively verbose, garbage collected, class based, statically typed, single dispatch, object oriented language with single implementation inheritance and multiple interface inheritance. Sun loudly heralds Java’s novelty.

2001 - Anders Hejlsberg invents C#. C# is a relatively verbose, garbage collected, class based, statically typed, single dispatch, object oriented language with single implementation inheritance and multiple interface inheritance. Microsoft loudly heralds C#’s novelty.

Joking aside, in the early ’00s, Microsoft wanted a productive and safe framework for writing Windows applications, which included interfacing with com and ole … things. This framework was named the “.net Framework”3 Named in association with a now largely forgotten marketing push to be hip and cool and online. If anyone asks, the way to be hip and cool and online is to prefix all names with “.net” in capitals. More seriously, part of the push for the .net Framework grew out of the need to distribute com things over the /net/work. I’m sure there was a beautiful vision there.. They could have written this framework in C, but this would have forced all applications to be written in C, which would be directly contrary to the goal of productivity and safety.

At first, they tried working around the issues with C, but it didn’t take long before they realised that working around the issues with C would be much harder than simply creating an entire new language with a runtime and everything.

So they did. C# was born as a “C with classes and garbage collector”, along with the .net Framework libraries. This is where things get a bit confusing, because for a while, C# and the .net Framework were essentially one product. You compiled C# code with the .net Framework, and then ran it on the .net Framework. It’s all a bit opaque and it only happens on Windows.

What makes this even more confusing is that the .net Framework was intended to be a multi-language environment: Microsoft made it run things like Ada, Cobol, Python, and so on, but this never really took off and I’m not sure how well supported those efforts are right now.

Standardisation

I don’t know why, but Microsoft decided it would be a good idea to get a standards body to sign off on C#. This happened in 2003, and Microsoft did this in a way that’s actually sort of good.

Obviusly, they had to un-opaque C# and the .net Framework. They split these apart into separate components, and created well-defined interfaces between them. In this spirit, there are actually two standards: one for the runtime platform and the other for the language.

Common Language Infrastructure

The cli (Common Language Infrastructure, iso/iec 23271:2012) describes three things:

  • the Standard Library4 Seriously, Microsoft, why doesn’t this one get a fancy abbreviation like the others? with data structures, methods for i/o, dealing with text, run-time manipulation such as reflection, and, crucially, xml5 It is the early ’00s, after all!’;
  • the cil (Common Intermediate Language6 Sometimes also known as .net il.), a platform-agnostic bytecode; and
  • the ves (Virtual Execution System), a runtime platform (the name gives away that this is a bit like the jvm, the Java virtual machine) for running the bytecode.

Where does C# fit into this? It’s surprisingly simple, actually. C# is a separate standard (iso/iec 23270:2018) that defines a language that compiles in a particular way down to the platform-agnostic cil bytecode. That’s it.

.net Framework

Of course, Microsoft were the first to market with an implementation of this cli standard7 They’re always one step ahead of their competition!. Their implementation was called the .net Framework. It has all three components:

  • the fcl (Framework Class Library) implements the cli Standard Library;
  • the msil (Microsoft Intermediate Language) implements the cli cil; and
  • the clr (Common Language Runtime) implements the cli ves.

By the way, let’s take a break here to make a note on nomenclature. There’s a general trend in the way these things are named: If something is called “Common” or “Core”, it refers to a concept in the platform-agnostic standard, whereas if it’s called “Microsoft” or “Framework”, it refers to a Windows-only implementation of the same concept.8 Which is also why the clr is abysmally named – it’s the Common Language Runtime that runs only on Windows.

It’s worth also mentioning that, at this point, C# is a completely separate entity from the .net Framework – but only on paper! The only C# compiler is still the one that ships with the .net Framework sdk. You can write C# code that doesn’t touch the .net Framework libraries and only rely on the bcl (Base Class Libraries), but it’s still compiled by the .net Framework, so it really doesn’t make a practical difference.

Mono

Given the actual standardisation of the cli, it didn’t take very long for someone to get involved in trying to create a Linux implementation of it. What did take time, however, was getting anywhere near feature parity of the Microsoft implementation. In the early days, the Mono project consistently lagged behind the Windows-only .net Framework implementation of the cli from Microsoft.

Things have improved, though, and Mono is quite capable (and has been, for some time now). It also has all three components:

  • Mono implements the cli Standard Library;
  • Mono implements the cli cil; and
  • Mono implements the cli ves.

As you can see, Mono went for a slightly more minimalistic naming strategy.

Mono also compiles C# code to cil, so finally, we have a C# that’s actually separate from the .net Framework.

Open-Source

In 2015, a Brave New Microsoft™ decided to be more open source. They acquired GitHub, they added a Linux compatibility layer to Windows, and they released the source code to large portions of the .net Framework. The .net Framework does still not run on _nwp_s, mind you, but at least now we can see the code that prevents it from doing so.

More importantly, it allowed for much greater information sharing between the Mono and .net Framework projects. In 2015, Mono started incorporating code from .net Framework, as well as contributing patches back.

At this point, we have two relatively similar projects that accomplish the same thing: compiling C# and running cli code. One project works on Windows, and the other project works on many platforms. It was inevitable that the synergy would take a turn for the incestous, and indeed Microsoft acquired the company behind Mono in 2016.

However, it would be bad strategy on multiple levels to try to wrestle the Mono brand under the Microsoft flag, so one more thing happened in 2016: Microsoft created its own cross-platform implementation of the cli, and called it .net Core. This implements the three parts of the cli standard with the following names:

  • Core_fx_ implements the cli Standard Libraries;
  • .net Core implements the cli cil; and
  • Core_clr_ implements the cli ves.

Aside from this, the .net Core sdk also compiles C# to cil.

You’ll note that this means that Microsoft is now in competition with itself, owning both major cross-platform cli implementations: Mono and .net Core.9 This reeks of Embrace, Extend, Extinguish and makes me rather uneasy. This is an unpopular opinion and I hope the masses are right, but I privately worry we’ll see Microsoft fencing in the .net ecosystem soon enough.

It’s also worth mentioning another natural consequence of this: while .net Core may seem like the more production-ready implementation with greater corporate backing, it is by far the younger implementation and I wouldn’t want to say anything about the amount of corporate backing.

All in all, we now have no less than three implementations of the cli standard:

  • .net Framework, the original implementation that spawned the standard, ships with Windows and runs only on Windows, and was closed-source until very recently;
  • Mono, the early open-source cross-platform implementation that nowadays share a lot of code with .net Framework and is, to a large degree, compatible with it; and
  • .net Core, the young attempt by Microsoft to create an open-source cross-platform implementation under its own brand, while taking liberties that break compatibility with the .net Framework.

About the compatibility between .net Framework and .net Core: it isn’t. Microsoft envisions .net Core as a newer, better version of .net Framework and takes the opportunity to improve massively on old cruft where it couldn’t before for compatibility reasons.10 And this is where I get a bit sad, because it as far as I can see, it seems like Mono does not support (nor has any plans to support) .net Core. This means, unless I’m misunderstanding things, I have to choose to either use the improvements in .net Core, or build and run my code using the mature Mono project.

Obviously, this leaves the Mono project in a bit of an uncertain future. It looks like the path ahead for Microsoft is to slowly deprecate both the .net Framework and Mono, and encourage people to use .net Core instead.11 This is indeed what seems to be in the works for the future release of .net 5, but it’s far enough into the future that I won’t read too much into it.

F#

Now that we’ve sorted out all the various runtimes, it’s time to make a brief mention of the best stuff: the F# language. If you are into this, I strongly recommend reading the history of F# recounted by Don Syme12 The Early History of F#, Don Syme, 2017. First draft available online., who has been a part of the journey since the start. Anyway, here are the important bits:

The division known as Microsoft Research was invited to participate in the design of the .net Framework. In a fun clash of controversy, despite being invited to participate, Microsoft Research is wholly and completely separate from the product divisions. They had some opinions that were at the time considered academic and impractical, but were still included in the .net Framework (and, by extension, the cli standard.) These ideas include real support for tail-call optimisation13 Functional languages on the jvm need to bend over backwards to get around the lack of tco on the platform., first-class functions14 Not by creating and allocating a new wrapper class, as Java does it., and non-erased generics15 Runtime-aware generics are the foundation of many newer C# features as well..

Yet, while F# has been very influential in the design of the .net Framework, the opposite is also true: F# was not intended to be a port of OCaml running on .net Framework – rather, F# was from the beginning envisioned to be a .net_-based functional language, with syntax and semantics inspired by OCaml. This is a subtle but important point of distinction. The hardest thing about interoperation between two languages is generally not the compilation to the common platform, but mapping data structures between the two languages without dropping performance through the floor. With the clear goal of being a _.net language, F# was frequently constrained to doing things however they were done in the .net Framework clr.

Being a product of Microsoft Research, F# has always had a sort of half-open status in terms of culture and community outreach. Initial releases of F# shipped with the source code and there has always been a continuous effort to make F# run on Mono. While it wasn’t until 2010 the F# code was truly released under an established open source licence, this air of open development that has followed F# is the reason it sometimes feels like there’s better support for F# than C# on nwp.16 At the time of writing, .net Core does have support for building and running F# projects, but the tooling is – as always – lacking. The best F# experience on _nwp_s are through Mono.

Visual Studio Projects

Completely orthogonal to the standards, there’s also Microsoft Visual Studio, which greatly influences how people work with .net code. In particular, Visual Studio stores important information regarding .net projects in project files, generally with a file extension like .csproj, .fsproj, etc., depending on which language the project uses.

It is far from trivial to compile a .net project without the information contained in those project files, which lead Mono to include a tool called xbuild which reads those files and compiles the whole project. These days, Mono shares enough code with the .net Framework to instead ship an msbuild executable for doing the same thing, and thus xbuild is deprecated.

Similarly, multiple Microsoft Visual Studio projects can be linked together (i.e. loaded/built at the same time) by listing them together in a Microsoft Visual Studio solution file. These files are also supported to varying degree by the different implementations of .net.

Active Server Pages

This text is way too long already, but I have to give this a brief mention as well. There has always been an asp.net framework for creating dynamic websites, shipped with the .net Framework, alongside with the libraries for creating Windows applications.

What has happened in the .net ecosystem, and the industry at large, is a gradual shift from component-based Visual Basic-style gui applications to web applications. In later versions of .net Framework, the main selling points are no longer the gui builder tools, but the asp.net framework. With .net Core, this is taken an additional step further: there are no longer any gui builder tools; it’s all about the web.

And with that, Microsoft has, in some ironic way, gone from 2001 over to 2019, and in a process of slowly correcting mistakes, begun fulfilling the initial goal of being hip and cool and online. They finally have reason to call it .net.

Referencing This Article