History of the VS IDE Doug Hodges is interviewed by Ken Levy discussing the history of the Visual Studio IDE (Integrated Development Environment). Ken: Who are you, what do you work on, where are we? Doug: Well, hello, I’m Doug Hodges. I’m the original architect of the Visual Studio IDE. Ken: Is that also referred to as the Shell? Doug: Yeah, the main shell for the Visual Studio product. All the language products that plug in are all integrated in the IDE (integrated development environment). Ken: What’s your exact title? Doug: I’m Software Architect for the Visual Studio IDE. At least that’s what I call myself. Ken: So we’re in building 41, which is kind of the home of Visual Studio. Doug: Yep. Building 41 in Redmond, Washington. Ken: So Jason Weber tells me that he refers to you as the grandfather of the Visual Studio shell. Doug: I prefer to consider myself as the father, not the grandfather. Ken: So he said that you consider yourself the cool uncle, is that right? Have you said that before? Doug: Yeah, well. So I’ve been on this product and this vision to create this integrated extensible IDE from starting back in the Visual Studio 5.0 timeframe. Ken: What year was that, do you recall? What version of Visual Studio? Doug: Visual Studio 97. And back then I was the architect of Visual InterDev (VI). And actually we started the product back in ‘95 when Microsoft changed its focused on addressing the Web. And we took my product, I was actually part of the Microsoft Access team, and we were working on a future version of Microsoft Access that was based off of the component technology. We had a forms-based technology package that ultimately became Internet Explorer as you know it today. We had the database design tools which were a component that ultimately became the database design tools in Visual Studio today. I was on a team that was suppose to take these components and repackage and rebuild a product that would be Microsoft Access, the follow-up and successor to Microsoft Access 2.0 at the time. I was an original member of the COM (OLE) team, and so we started with a COM-based strategy for how you could integrate components. When we [got] a new charter to build a tool to address developing for the Web, we created this Visual InterDev product. We took my team and another team that was doing development tools for Blackbird, -it was for MSN-and we formed a team to create Visual InterDev. At that time we had one tool for VB (Visual Basic), one tool for VC (Visual C++), and at that time we thought that the VC codebase was going to be the codebase that was going to evolve into an integrated shell. We built our first version of that Visual InterDev product in the VC codebase, and I re-hosted those same components and forms package for HTML, design, and database tools in the VC 5.0 shell. That was the start of our component (COM) interfaces that you see still existing today. We had a vision in the division to create one common IDE for all language products and all tools. We had a separate IDE for VB at that time, then we had VC, VJ, and VI in this dev shell, the VC-based shell. And it was a derived MFC-based classes implementation of a C++-based framework. We tried to have a third-party extensibility program like VSIP (Visual Studio Industry Partner) program today. We found that wasn’t going to work very well unless everyone gave us their source code that we would use to build in our own build lab, because any time you change something in a base class you had to recompile the world. So that wasn’t going to work to have a third-party extensibility program. So then in the Visual Studio 6.0 timeframe, I started this new shell and the whole strategy was based on a COM-based architecture that could allow independently built things that were independent versions of each other to be plugged together. In our first version, we took the VJ++ product and the VI product out of the MS Dev shell, and they were the first two products that we integrated into this new IDE that we created. So at that time it looked like we took a step backwards, because we went from two shells, one for VB and one for VJ/VC, and then in the VS 6.0 timeframe we moved to three shells with one for VB, one for VJ++, and one for VC++. But that was a stepping stone to get us to the goal of our division which was a single shell which we achieved in the Visual Studio 2002 (VS 7.0) timeframe. Ken: So at this stage of the game when you’re building this third but new one, how are you thinking about extensibility? Is it mostly around certain partners in the VSIP program, or was it about anyone doing anything they want at any time? Doug: The Shell was designed to be fully extensible from the ground up and designed to have independent pieces that independently plugged together. We had all of our existing products internally, and we wanted to be at least as useable as the VB product and the VC product combined. We picked the best features from those two products, and we wanted to meet the extensibility requirements of those independent teams. The VB team, the VC team, and the VI team were all customers of mine, plugged into this combined IDE as well as planning for the requirements of third parties. Yeah, it was designed from the beginning to be a fully extensible shell and we felt that if we could meet the requirements of VB and VC, that together we could probably meet the requirements of most language partners. And then of course you had to anticipate the kind of other products that could be built, like what you see today with Team System or Server Explorer for database development, and other things. Yeah, the goal from the beginning was to be extensible. Ken: So you are targeting both languages to be integrated as well as UI tools and components. And sometimes they are separate-you may have one company that wants to build a new language, like Cobol or something, and another company that has a toolset that is for a specific vertical market and they want to resell that integrated into Visual Studio, is that right? Doug: Well, yeah, and trying to cover the full lifecycle of development tools to assist in not just code development but in testing, deployment, servicing, source control, and code analysis. Ken: The companies that did stuff in the beginning were basically partners-they got access to the team-and it didn’t scale to the masses. Doug: Originally, these were the same APIs that we used to build our own products, and we opened it up to a select program that you could pay to get into. It used to cost $10,000. Those where the high-end ISVs at that time. Very often it was a high end C++ programmer who worked on it, and priority was given to powerful features but not necessarily on making it easy to use. At that time we had no VS SDK team and I had to beg and borrow to get resources to get a sample here and there or documentation. Originally when someone needed to write a feature for the shell, I would help them with a little planning of their product and they would write a sample and send it to me. Some of the samples we had at the beginning came from different sources and we didn’t have a team to create or organize an SDK. It was a very small program at that time. Another aspect of extensibility was add-ins and macros, and those were in the product in the beginning, and every box of Visual Studio had wizards so if you wanted to write add-ins you could do extensibility right in the box. That was more targeted more at VB-level languages using an automation-based model. But you were limited in the kind of features that you could implement there. You could add commands or tool windows, or hook to certain events and react to them. But you couldn’t implement a new language, or a new debug engine with a custom runtime. That’s when you needed to access the lower level APIs, and originally, like I said, it cost $10,000. Ken: Did they get code as well as help? Was there kind of an SDK for VSIP folks? Doug: There was a little bit of an SDK. Originally there was a set of samples I scrounged together by begging and borrowing. We had someone in PSS come and help us write some samples. They helped to build the tools you needed-the IDL and .H files, and other related tools that you needed. There wasn’t a nicely packaged SDK at that time. Ken: Did you end up working in some capacity with most VSIP folks, and you knew them in some way from e-mail or a demo? Doug: Yeah, I worked with them. We started way back having these developer labs where we brought partners in house for week long developer labs. Those started up way back in the 2002 timeframe. Originally that first extension of the shell was in the VS 6.0 timeframe and was not yet open. We did not expose the extensibility APIs. We had a couple of internal teams write some things, and I think someone from the SQL division wrote something for the actual language query or something. In the 2002 timeframe we basically had developer labs were we could bring the partners in house, and they got a lot of hand holding. Ken: How many times have you given the overview presentation for Visual Studio extensibility? Doug: <laugh> Yeah, I can’t count how many times. Practically every developer lab started out that way with my overview presentation. Yeah, gosh, I’m not sure. A hundred? Ken: Where are we today? How did we get to where we are today? Doug: So with that first version we achieved our goal of a single integrated IDE in 2002. We shipped the 2003 and 2005 versions and are now working on the Visual Studio 2008 version. Now we can enhance and refine the product with new scenarios of Visual Studio extensibility, and it’s a very mature platform now. Ken: So a few things happened over the last few years. There is now a VSIP level that is less than $10,000, and then in 2005 the VS SDK became free, so that basically anyone can download the SDK and extend Visual Studio, although they don’t have the same co-marketing or complete technical support access, but yet it’s free so anyone can do it. Doug: Yeah, yeah, that’s really cool. So from a technology point of view there is a free level with access to all of the technology, then two pay levels above that which have additional marketing and support benefits. When we went public with the SDK, there were many many downloads. Then that raised the priority to not just worry about C++ programmers that have direct access to dev labs and hand holding with Microsoft. We needed to start focusing on the accessibility of the SDK and the accessibility of how easy it is to do that development. So that’s been a major goal over the recent years, to try to improve the quality of our SDK. We developed an actual team to own the release of the SDK (VS Ecosystem, or VSX team). Ken: We kind of had two different groups out there. First we have VSIP, the partners, which are part of the program, who are extending Visual Studio usually to a product that they can sell or use in the enterprise. And now this new group that we’re cultivating called VSX, which stands for Visual Studio Extensibility, which are essentially non-partners who are using the free VS SDK, and that’s a fairly new thing. Is this something that you had envisioned a long time ago, where you see .NET developers, Visual Studio developers, coming in and building all kinds of stuff for CodePlex and other places? Doug: The original goal of focusing on the high end ISVs shaped the style of APIs we have. It was targeted at making things possible. Now that we have released the technology to everyone, we need to greatly improve our level of abstraction and ease of development for some of these scenarios. I can’t say I originally thought that we would have that available. We had myself and a program manager trying to run a program that doesn’t scale well. But now that we have a full product unit around managing and growing this program, it’s really exciting and I hope it goes and develops really well. Ken: A moment ago you touched on ease of use. In addition to more functionality and capability for VS extensibility and the VS SDK, it’s also about making what’s there easier to use and access, whether it is simply better docs, or better sample classes, or better wrapper classes that get you in, or wizard-type functionality, right? Doug: Right. The first thing that made it easier was finally delivering the ability to use managed code to write these extensions. When we first shipped the extensibility via the SDK, you could only use C++ with it. We shipped both Visual Studio 2002 and 2003 that way. But we knew lots of developers would want to write in managed code like C# and VB. So we did what we called the Everett extras release. Everett was the codename for Visual Studio 2003. So we had an Everett extras SDK release which was an add-on to the previously released VS 2003 SDK release that delivered the minimal pieces you needed to be able to write in manage code. It gave you the COM interfaces in managed code, a redistributable MSI that you could use to install those assemblies, and then the beginning of a helper framework that you could use as some helper classes to begin to write something. That was really thin and we put a lot of effort and massively grew those helper classes into what we call now the Managed Packaged Framework (MFP) in the Visual Studio 2005 release. So we continued to build and always provide the interop assemblies, and one of the things that has always been important to me was designing the APIs to take into account different extensions that different companies will work with at different times and different schedules. Addressing versioning has been a particular focus, so we spend a lot of effort maintaining compatibility with old versions of Visual Studio. It’s always been our goal to maintain compatibility, and the API is designed in a fashion that you could write a single extension that runs in Visual Studio 2003 and Visual Studio 2005 that runs with a single binary. More realistically between Visual Studio 2005 and Visual Studio 2008; that’s going to be popular, that you can create a single binary that runs in both versions. So part of what we do is to make a big effort to not change or break compatibility with any previously shipped interfaces. We create new APIs in a way that is defined in a new assembly and file, so you can see what is new in a version. If you need to, you can provide the new distributable that lets you install the interop assemblies for the new interfaces on a down-level machine. If you write a package that depends on new features you can run the new version and can take advantage of the new features. But that same binary can run a down-level version, so that query interface or query service for an API can allow you to dummy down and write code to work without it. That’s particularly interesting to high-level product development, like partners who release VSIP products that may address more than one version of Visual Studio. Someone writing a tool for themselves probably won’t be worrying about that and will only be worrying about one version of Visual Studio. Ken: What are the key prerequisites for someone who wants to download the VS SDK? Say someone has Visual Studio 2005. Do they need to know a bit about COM interop or a little bit about anything else? Because there are parts of the shell that are pure COM. The Managed Packaged Framework (MPF) helps a .NET developer access them. Doug: So we have this Managed Packaged Framework that covers and abstracts, let me say, 30% or 40% of our APIs. And so if you are staying within that subset, you can see what looks like a .NET experience when writing your extensions. But if you are doing something like writing a language or interacting with, oh gosh, there are so many features. You are going to come upon areas that are not yet abstracted in the Managed Package Framework, and then yes, you will need an understanding of COM interfaces. Unfortunately in COM interop there are several places you are going to encounter an end pointer and then will have to deal with the marshal class, the marshalling, and deal with an end pointer into an object or a string. We have samples for those, and hopefully when you hit one of those areas you can find some sample that does something like it so you can copy from it. And that’s one of those areas we hope to improve the coverage of the MPF and other techniques to eliminate those places where you have to directly go to a COM interface that was originally designed for a C++ programmer that is rather awkward to deal with from a managed language. Ken: One of the things that we’ve seen recently is the community doing a bunch of cool stuff building on top of the SDK. For example, the VSSDK Assist on CodePlex.com. You’ve seen a demo of that recently. Doug: Yeah, that looks really cool. That looks really awesome. Ken: How would you describe it? Is that something that assists people that use the SDK? Doug: It’s leveraging the Guidance Automation Toolkit that was developed in combination with Visual Studio people and the PAG (Patterns & Practices Group). And it’s really cool. It’s a factory. It’s a wizard, on steroids. You can start with a new project wizard that starts you off with writing something for Visual Studio extensibility, but it doesn’t drop you out of the wizard when it’s done. It’s like fine-grained points to launch wizards to add additional features to extend your project and give you guidance on what’s next. It gives you help. Like if this is what you’re looking to do, it can give you help on how you can do it. It’s a direction that’s fantastic. It’s a direction that I’d like to see continue to grow. Ken: One thing that I noticed in the demo is at runtime after you run it, it does not require any components at runtime. It just generates code in the right places that just use the VS SDK and extensibility APIs. So there’s nothing at runtime that you need; it’s just helping you and assisting you learn the SDK. So if you want a tool window and a button, here’s how to glue those together. Doug: And it knows about editing the five places you have to edit, you know, this file over here and this file over there, with one UI sequence. It takes care of making all those edits. But you’re right, it’s not a new runtime you install to use it; it actually generates the code for you. Ken: What are some things that developers might be capable of doing if they were to download the SDK? Things like creating services, tool windows, menu buttons. You can even just have a simple thing that is on a menu that launches your own application. What kind of things can they hook into? What would you like to see the community build? Doug: The first simple thing is adding a command on a menu or a toolbar. That command can do some action on selection, and maybe the next thing is showing some UI in a tool window. And let’s say you have your own kind of code review tools and you have your own tool window that shows the results of a code review, and your command launches your code review based on the selected code file and presents the results in a window. Now incidentally, that level of extensibility, where you are adding command and tool windows, that’s something you can also do using add-ins. And the extensibility that’s been in the product for free since the beginning. In many ways it’s easier to use add-ins, so I don’t want to discourage people from using add-ins. There are some subtle features that you can’t do in a tool window, for example through an add-in, that you can do when you are coding at a package level. For example, if the user shows a tool window and then shuts down, the next time they launch Visual Studio the tool window automatically reappears. |