Domain-Specific Languages (DSLs) and Software Factories


November 22, 2005

Click Domain-Specific Languages (DSLs) and Software Factories to listen to this ARCast.

Ron Jacobs:   Welcome! Hey! Welcome to Arc Talk. This is your host, Ron Jacobs, and I am so delighted that you are with me today, because today we are going to be talking about, you know, just one of those crazy things you hear about and not sure if it is something you need to tune into.

I don't know if you are like me but, you know, around Microsoft we hear about a lot of stuff, happening stuff, that's coming, and sometimes you are, like, "Well, I just can't listen to everything." So, you turn off something and maybe you have heard about people talking about DSLs, and we're not talking about the communications technology, but rather we are talking about the domain-specific languages, or maybe you have heard them talk about the software factories on and, and in this kind of stuff, well, if you thought this is something you didn't need to know about, you were wrong, my friends, you are so wrong.

And today on the podcast, we are joined by Steve Cook, who is just a great guy! He called me from the U.K. and I just hooked up the phone to the recording stuff, so we can hear from him, and he has got some great thoughts on what this is all about.

I think you are going to enjoy listening to Steve and learning how this could apply to you and, to tell you the truth, I am now going to have to spend some time, like, trying to create my own DSL thing. In the past, I thought, Now that's for somebody else to do, but now I am thinking really, there's probably good reason to do it.

So, you tell me what you think. Right, tune in, listen up, we are going to have some links at the end of the podcast, and you can go to these links and discuss what you think. We have a wiki page, where we can write up some, some thoughts about DSLs and software factories. Anyways, it's cool stuff! Listen up! Steve Cook!

Today, I am joined by Steve Cook, who is a software architect in an enterprise tools group and, actually, Steve you are based out of the U.K., aren't you?

Steve Cook:   That's right.

Ron Jacobs:   And that must be, that must be challenging for you to work with us, with the Redmond guys, and being based out of the U.K.

Steve Cook:   Well, there is eight-hours' time shift, and often I work quite late at nights, but on the other hand I get in quite late in the morning, so that's fine, and I get quite a lot of quiet time during the day. OK. (Laughter)

Ron Jacobs:   All right, well, today, Steve, we are gonna talk about model-driven architecture and a little bit about what this is all about and what Microsoft is doing in this area, and so I think it's kind of an area that really I don't know a lot about, but I was looking over the OMGs page on model-driven architecture and it seem to me that they were sort of saying a story that we have heard before, which is essentially that, if you can describe your application in terms of diagrams and stuff, that there would be tools that you would convert it into any given platform that you might want, and so I reckon their goal of diagram they are saying is, you know, as long as you can describe your thing with UML and MAF and, and CWM, that they could then pick tools that might translate it into a Java app, a .NET app, a CORBA app, or whatever. So, what do you think about that kind of vision?

Steve Cook:   Well, MDAs—model-driven architecture, which is a trademark of OMG—actually means quite a lot of different things to different people, I think. And I was involved, very much involved in the OMG before I joined Microsoft. I represented IBM on the standardization of UML tools, for example. So, here are the few things that I think model-driven architecture means to different people.

First off, I think a kind of out there, in general, particularly don't know much about it. It just means model-driven development in general is to create some model and we generate some code from it, kind of thing. Then, a kind of more specifically, from the OMG's point of view, it means doing that using the OMG's technology. And even that means different things to different people, because there are some people, they talk about what they call platform-independent model and platform-specific model and transformation between these models.

So, as you said, you can propose, just propose you can build a model which is platform-independent or somehow transform it into like a Java application or .NET application. Umm, some other people think of it, that as UML is a programming language and there are some books out there, a couple of books out there really proposing that you think of UML as a programming language, and then I think some other people say, Well, it's not really about UML, it's about using OMG technologies to do model development. So, it's kind of hard to pin down what MDA really means.

But, I mean, I think I think you are right in categorizing it as being… If you like, I would turn to the cliché vision of "We will take models and will generate application from them," and I even think if to take that seriously you really have to question why case didn't turn out to be terribly successful, because I… I don't think, you know, having said what we said, that we really said anything that would make you any more successful with time around.

So, I think there are really important missing factors in, you know, the formulation of the model-driven architecture that's on the OMG Web site, all that we discussed so far, which will make it work. One of them is, you don't start with a model. If you take this as your starting point, are we going to have a model and we are going to generate things from it? That's really the wrong place to start. What you got to start from is what you are going to generate.

That's the absolutely critical thing to understand, that you are going to make any success of generating code from models and that's what we call the domain-specific development, which is a kind of different thing from MDA.

Ron Jacobs:   OK. So, let me ask you a question then about that, so is that if the starting point is not the model but the thing you are going to generate is like the application, I guess, is what you are saying?

Steve Cook:   Well, part of the application. Yes!

Ron Jacobs:   OK, because, well, it seem to me that one of the, one of the weaknesses of, of this model, at least I found it sort of frustrating in doing UML stuff is that it tries to be all, so all-encompassing and general-purpose that maybe domain-specific will allow you to be more specific to a kind of problem you are trying to solve. Is that what you are getting at? That this is the big advantage where we are proceeding?

Steve Cook:   Yes! Yes! There is a kind of basic pattern here which is… all I think of is, being the language-driven programming pattern. This again has been around for while, but really does it if you have a framework, a bunch of code in a framework, then it's a particular kind of code you write to use that framework which I think of as being configuration code. It's a code that takes the framework and instantiates it to your particular case.

You know, you create instances of this and instances of that and wire them up, and that's the solution to your problem. And you see a lot of that kind of thing. For example, you see that you are creating GUIs and you create controls, forms, and stuff and wire them together through configuration code, and then you get the similar pattern of applying in many domains, and I think that's the kind of code you can target your generation at.

That's the kind of code you want your models to transform into, so there is a really intimate relationship between the models and the framework, which is, the models are essentially creating configuration code for the framework.

And if you take that perspective, then what you want to do is design a modeling environment specifically for that job and specifically for generating for that particular framework; and if your starting point is UML which, as you say, is very kind of ambitious and trying to talk about lots and lots of things (and, if I might say so as an author of it, is very kind of up and down and patchy in the level of abstraction which it talks about things), then if you are trying to address any particular configuration problem with it, then you will find that it doesn't fit.

You are going to have to somehow bend it to make it fit. You are going to have to put extra labels and spell your tags and expanding things to the UML, and build special generators that understand those labels and patching things, and you are up with something which fits together with the framework. But, you know, that can be a big job. It's not for the fainthearted. UML specs in itself is, you know, hundreds and hundreds of pages long. It may be easier just to take the job of generating this special configuration code, design the language to directly to build it, and that's the view we take. It is easier to do that.

Ron Jacobs:   You know, so, I can completely understand what you are saying, because it seem to me that when I first learned UML, about '95, that it worked really well for describing the object interactions like within, you know, a business-logic layer and, you know, I could really use it very well for that sort of thing; but as I tried it to use it to describe more complex, distributed applications or, you know, even things like Web services later on, when they came around, it seemed harder to get important elements of the model into the UML, because it didn't really have a construct for describing the element of the model that you wanted. So, you always end up kind of fudging the model some way to make it work.

So, I can completely get that and I think, like you said, that understanding the framework you are targeting, it's almost like you would develop a very different kind of modeling language to describe a GUI-based app versus maybe a collection of Web services that are going to do something, that have nothing to do whatsoever with UIs. The models might be entirely different for the way that you described the thing you are intending to build.

Steve Cook:   Absolutely, I mean, yes UML was designed in the 1990s. The kind of prevailing methodology of the OOA&D—object-oriented analysis and design—where there was a theory going around at that time that you can do object-oriented analysis.

You kind of analyze the objects in your business domain and then you build, kind of, sequence diagrams to depict how these objects talk to each other, and to get your, if you like, the sketch of the behavior of the software, at a kind of a high level where you haven't tried to map it down to any particular technology.

Then… The flip-back then was, you could kind of add detail to that model and turn it into design, and actually, eventually turn it into the code. Now, these days, you know, it just won't work, right! You can't add details to turn an object-oriented analysis model into a bunch of collative Web services or distributed services with, you know, load-balancing going on in between them and all that interesting stuff, right! It just doesn't work like that. We live in a different world these days.

So, I mean, basically, I think lots of people think of UML in that way that you described, which is you got class diagrams, you got sequence diagrams, you got use-case diagrams, maybe you got state diagrams, maybe you got activity diagrams, but really those three like classes, use cases, and sequence diagrams and you do, kind of, a high-level object oriented analysis, as a documentation, as an understanding of your problem; and then, by and large, you throw it away, right! You have used it to understand your problem. You then got to move on to mapping it to today's technology.

Now, if you look at some of the tools we just put out in Visual Studio 2005 of the application designer, that's a modeling tool for designing applications that talk to each other, right! And it has rectangles, with ports on the edge, and lines. Those ports on the edge of the rectangle represent end points, lines represent communications, and the boxes represent applications. Well, there is something in UML like that. The thing about our modeling tool is that the model is basically directly connected to the framework. Umm, in our case, in our particular tool, ASP.NET implements this particular stuff, so there is no big conceptual leap between the model and the framework.

Its just direct mapping. Umm, I think there it's not an important distinction to make about UML apart from that that it's really object-oriented analysis design language, if you like, and it should be checked as that; I mean, it doesn't work with the other things. The other distinction to be made is… but most people understand it through notation.

It's extremely popular as a notation and it is not popular as a meta model. But if you people understand it as a meta model and if you want to use it for model-driven development, you have to understand it as a meta model.

And then you really get into some kind of deep stuff, you know, where I think it is just not a very efficient place to be today. You can make it work, but what you said, you know, it really doesn't fit really well. So, what we think is, you should take to notation as a brilliant inspiration, right! Where people expect to see UML notational convention, like the way it organizes, you know, the direction of inheritance arrows or the places that black diamonds appear, all those things that people kind of understand. Let's reuse all of that. Let's, you know, not reinvent any of that. Where it's really not the right tool for the job, let's not try and use it.

Ron Jacobs:   So, you know the, interesting thing about what we've done, I think, you guys have done (because I didn't have anything to do with it but on VS 2005) is, rather than just building a better modeling language, we actually built a platform for domain-specific languages with the idea that anybody ought be able to build one of these.

Steve Cook:   Yes, that makes it easy to build the language. We talk about language-driven development or domain-specific development and the basic pattern is that you start by identifying, in your domain, or your framework, or your problem—whatever you want to call it—some holes that you can fill by defining models, which in the case of distributed applications could well be the holes you couldn't fill, could be the configuration of the applications and the way they talk to each other.

Right, so in the case of real-time system, the holes you might fill would be, you know, the state machine that runs that thing or… There is lot of examples, right! So, you start by identifying the hole that you want to fill, and you design the language to fill that hole.

We have got a framework that will implement modeling languages so basically, and also we bootstrap it, right, so we have a language for defining languages and we have a framework for defining languages, and the hole you fill in that framework is design of the language as a model.

So, we have a model for creating languages which includes creating the conceptual model for the language, the notation for language, serialization format for the language, the way the validation is done for the language, the way the code generation is done from the language.

So, you can kind of define all that quite quickly and deploy that and run it, and there is your language implemented as a tool in Visual Studio, and it will fill the hole in your framework. So, we are putting lot of effort into making it very easy to define new languages, and that's what we call the DSL tools.

Ron Jacobs:   So, you know, let me think through an example of this with you, for example, the group I used to be with, the patterns & practices team, has built a UI framework, they call it the Composite UI Application Block, which is a framework for building UIs and using a composite pattern with smart-client applications and, like you said, this framework, when you're targeting an application of this framework, there is a certain set of things you got to do, which is, like, building up the configuration that the framework is going to apply at run time.

So, you got to describe the composite pieces, these things we call smart parts, and the workspaces that are going to work together and how they relate together.

And you describe this partly in configuration, but partly in code, and so right now, though it requires the developer to just, kind of, know what this configuration stuff has to be like, through a lot of experience and then they can sit down and apply it, but if they were to take this extra step, they could build a DSL language with shapes and stuffs like that, and design surface that you could drag these things around and connect them up graphically, and then generate the working config code that will, that would make that thing functional.

It will probably still need code, but you would maybe handle, I don't know, 60-70 percent of the stuff that needs to configure the app. Is that a good analogy?

Steve Cook:   That's exactly right, yes, we haven't done that with the composite block that you are talking about, but we certainly could. We actually did a version of that with an earlier packed blockage of the UIP block, right, a User Interface Process block. We built a little demo language, which is essentially a flow chart and from that flow chart you would generate several things.

You generate the ad-hoc config file which contains the essential kind of logic of how you flow through the thing, and you generate some default pages which would then, you know, customize your serving code and generate some other stuff, I mean, we used templates to generate these classes and config files to complete the UIP block. Absolutely, you could do the same with the smart-client one, I am sure!

Ron Jacobs:   OK. So, to be practical, I know a lot of the enterprises I talk to, they have, you know, especially in larger ones, they will have a team of people that are like their .NET center of excellence, or something like that, and the idea is that this team of people would evaluate what kind of frameworks and libraries and things we are going to use in our enterprise, and develop best practices and training and what-not.

So that, the goal being that the business units who are building applications would build them with consistent frameworks, and that sort of thing. They would achieve more reuse and better productivity.

Is it out of the realm of reality for a team like that to build the DSL? So, maybe like, you know, you are working at some really big bank and you got your own kind of in-house framework that you have developed for building stuff. Would it be impractical to say, "Hey, you, at that bank, you could build your own language, your own DSL language to assist your business units when they go to build the apps with that thing"?

Steve Cook:   We've got several customers doing exactly that with the early version of DSL tools, at least evaluating them to do exactly that job, exactly that!

Ron Jacobs:   OK.

Steve Cook:   Absolutely, that's the key scenario for us.

Ron Jacobs:   Well, so that it sounds like that's where, you know, this thing could really take off, because, you know, a lot of people thought, Oh, well, yeah, maybe ISVs will go out and build these, but your average guy can't hope to figure that out, but it sounds like it's not that hard.

Steve Cook:   I actually… I think ISV is… The thing about ISV is they tend to probably want to start at a bit lower level. They may well want to use our modeling platform, but they don't really want to build a DSL in a day or even a week. They want to do something which for them is a product which has a lot of added value, so they are going to be operating at a more detail level of customization.

What we really are talking about is groups, just like the one you mentioned, or SI, like people who are running consultancies, who want to be able to reuse their own domain knowledge over a bunch of their customers. So, they might walk in with their own frameworks or their own DSLs, and be able to solve particular classes of problems in vertical industries quickly, because they have captured domain knowledge in their own language. So, our customers showing the most interest in what we are doing today are central architecture groups, large enterprises, and SI.

Ron Jacobs:   Now, well, I could imagine, though, like in the late '90s I used to work for ERP-type ISVs, and I could imagine those guys thinking of their ERP platform as really a highly customizable framework, because essentially that's what we tried to build, because every customer had a different thing and, you know, the way we would do it is when the customer purchased the stuff, we would send an army of consultants in there who would know how to customize our stuff to make it work for them.

But I could imagine that this would be a very powerful paradigm for even an ISV and that kind of model, and that you think of the components of your enterprise software as a set of framework, a set of building blocks that will need to be assembled, and giving them a DSL to assemble them would really be a helpful thing.

Steve Cook:   Oh, absolutely yes! I am not saying they wouldn't be interested, but all I am really saying is that they've probably got more funds to invest in that, so they might want to go into the platform a little deeper; but, absolutely, we are talking to ISV about several… actually, about the DSL tool, as well, for exactly that reason.

Ron Jacobs:   OK, so...

Steve Cook:   And I think another class of customers, rather, are actually individuals, right! It's so easy to build the language, but you couldn't just kind of do it. It maybe… take you a morning, and if you have got a task of problems to solve and you are an individual developer, it's worth doing that.

Ron Jacobs:   OK. So, if some, if somebody was interested in, in taking this up and doing it, I know I hate getting links over audio, but maybe what you can do is, Steve, is e-mail me some links and I will post that up with this recording on the same page, and so people can go and they can download the DSL toolkit and can play around with it and do have some resources to help them?

Steve Cook:   Absolutely! Well, there is one link which is kind of a top-level link for what we call a workshop on the Visual Studio site, and it's got links to our tools, it's also got some links to information about GAT and… and other stuff about white papers and software factories and all sorts of stuff, so I will send you that link, it's a bit tough.

Ron Jacobs:   OK, great! Hey, this has been really cool, and I am finally kind of getting the picture. Thanks a lot for taking the time to talk to us today.

Steve Cook:   Oh, you are extremely welcome.

Ron Jacobs:   Yeah, I told you, didn't I? Just tell me, yes, say it, say it out loud, say, "You are right, Ron. This DSL thing is very, very cool. I am just totally blown away. I have been around here, and I didn't get this." But, you know, hey, that's the way it is. There is so much stuff. We have to tune out something, because you can't possibly keep up with that all.

But now you are gonna want to pay attention to this, because software factories is very cool. Hey, so I haven't had a chance here to work up a cool new Web address for my new home here on the architecture-strategy team.

Maybe I will get something like wack archtalk. I am gonna ask for that, but until then you just have to go to the links here on the podcast page and look for the architecture wiki on channel 9 and, yes, it's an ugly thing to read a URL, but this is it, OK! Here we go:

OK, go there, that's my little corner of channel 9, and we have wikis where we're gonna just build up knowledge and share with each other, because you feel so much better when you do. And we'll see you next time on Arc Talk.