Blogs

SpringSource Blog

Better DSL support in Groovy-Eclipse

Andrew Eisenberg

The Groovy language is an excellent platform for creating domain specific languages (DSLs). A good DSL can make programs more concise and expressive as well as make programmers more productive. However, until now these DSLs were not directly supported by Groovy-Eclipse in the editor. When DSLs are used heavily, standard IDE features like content assist, search, hovers, and navigation lose their value. For a while now, it has been possible to write an Eclipse plugin to extend Groovy-Eclipse, but this is a heavy-weight approach that requires specific knowledge of the Eclipse APIs. Now that Groovy-Eclipse supports DSL descriptors (DSLDs), supporting custom DSLs in Groovy-Eclipse will become significantly easier.

A simple example

Consider this DSL described by Joachim Baumann. He creates a simple DSL for working with distances. Using this DSL, you can write things like this to calculate the total distance travelled:

3.m + 2.yd + 2.mi - 1.km

This is a simple and expressive DSL, but when you type this into a Groovy Editor in Groovy-Eclipse (for conciseness, assume that $url is defined elsewhere):

Custom DSL not recognized in Groovy-Eclipse

You see underlines and no hovers, meaning that the editor cannot statically resolve the DSL's expressions. Using a DSLD, it is possible teach the editor some of the semantics behind these custom DSLs as well as provide documentation for hovers:

DSL in editor with documentation and no underlines

To create the DSL descriptor for the distance DSL, you simply need to add a file to your Groovy project with a .dsld file extension and the following contents:

currentType( subType( Number ) ).accept {
   property name:"m", type:"Distance",
    doc: """A <code>meter</code> from <a href="$url">$url</a>"""
}

This script says whenever the type currently being evaluated in the editor is a subtype of java.lang.Number, add the 'm' property to it whose type is Distance. The currentType(subType(Number)) piece is called the pointcut and the code block with the call to property is called a contribution block. More on these concept later.

This script snippet above is not the complete DSLD. It only adds the 'm' property. To complete the implementation, you can take advantage of the full power of Groovy syntax:

currentType( subType( Number ) ).accept {
    [ m: "meter",  yd: "yard",  cm: "centimerter",  mi: "mile",  km: "kilometer"].each {
      property name:it.key, type:"Distance",
        doc: """A <code>${it.value}</code> from <a href="$url">$url</a>"""
    }
}

This simple example shows that a relatively small script can create some powerful DSL support.

Anatomy of a DSLD

DSLDs enhance Groovy-Eclipse's type inferencing engine that runs in the background while editing. DSLDs are evaluated by the IDE and queried by the inferencing engine as necessary.

A DSLD script contains a set of pointcuts that are each associated with one or more contribution blocks. A pointcut roughly describes where type inferencing needs to be enhanced (i.e., which types in which contexts) and a contribution block describes how it is enhanced (i.e., which properties and methods should be added).

Many pointcuts are available and they are described in detail with examples in the DSLD documentation. The set of available pointcuts is likely to expand in future versions of DSLD as we start to learn how people will be creating scripts and what kind of operations they require.

Contribution blocks are Groovy code blocks that are associated with a pointcut through the accept method. The two main operations you can do inside of a contribution block are property, which we have introduced earlier, and method, which adds a method to the type being analyzed in the contribution block.

The term pointcut is borrowed from Aspect-Oriented programming (AOP). Actually, DSLD can be considered an AOP language. The major difference between DSLD and typical AOP langauges like AspectJ is that DSLD operates on the abstract syntax tree of a program being edited, while languages like AspectJ operates on the Java byte code of a compiled program.

Getting started with DSLDs

There is full DSLD documentation on the wiki at Codehaus. Here, I will briefly describe how to get started with DSLDs. To get started:

  1. Install the latest nightly build of Groovy-Eclipse using this update site:
    http://dist.codehaus.org/groovy/distributions/greclipse/snapshot/e3.6/
  2. Inside of a new or existing Groovy-Eclipse project, copy the DSLD
    Meta script into a source folder of the project. This script provides
    editing support for DSLD files themselves and is available here
  3. Create a new DSLD script using the wizard: File -> New -> Groovy
    DSL Descriptor:
    DSLD Wizard
  4. In the newly created file, uncomment the sample text.
    currentType(subType('groovy.lang.GroovyObject')).accept {
         property name : 'newProp', type : String,
            provider : 'Sample DSL',
            doc : 'This is a sample.  You should see this in content assist for all GroovyObjects:<pre>newProp</pre>'
    }
    

    Inside the DSLD you should see content assist and hovers specific to DSLDs (this comes from the meta-DSLD script added in step 2). It will look something like this:
    Contents of DSLD file with hover

  5. Now, you can create a new Groovy script and play around with the DSLD you just created. You can type:
    this.newProp
    
  6. You should see that newProp is properly highlighted and that hovering will show the documentation from the DSLD, and it should look something like this:
    Using the sample DSLD in a file

  7. You can make changes to the DSLD. After saving the changes will immediately be picked up in all your Groovy scripts and files.
  8. Congratulations! You have now implemented your first DSLD.

You can view and manage all of the DSLDs in your workspace from the Groovy -> DSLD preference page:
DSLD Preference page

From here, you can enable/disable individual scripts as well as choose which scripts to edit.

Important: since finding and fixing errors when implementing DSLDs can be a bit cryptic, it is strongly recommended that you do the following:

Compile and runtime problems with your script will be shown in one of these two places.

A DSLD for the Grails constraint language

For a larger example, let's look at the Grails framework. The Grails constraint DSL provides a declarative way to validate Grails domain classes. It is clear and succinct, but without direct editing support for this DSL, Grails programmers rely on external documentation and may not be aware of syntax errors until runtime. We can create a DSLD to solve this problem

// only available in STS 2.7.0 and above
supportsVersion(grailsTooling:"2.7.0")

// a generic grails artifact is a class that is in a grails project, is not a script and is in one of the 'grails-app' folders
def grailsArtifact = { String folder ->
	sourceFolderOfCurrentType("grails-app/" + folder) &
	nature("com.springsource.sts.grails.core.nature") & (~isScript())
}

// define the various kinds of grails artifacts
def domainClass = grailsArtifact("domain")
// we only require domainClass, but we can also reference other kinds of artifacts here
def controllerClass = grailsArtifact("controllers")
def serviceClass = grailsArtifact("services")
def taglibClass = grailsArtifact("taglib")

// constraints
// The constraints DSL is only applicable inside of the static "constraints" field declaration
inClosure() & (domainClass & enclosingField(name("constraints") & isStatic()) &
		(bind(props : properties()) & // 'bind' props to the collection of properties in the domain class
		currentTypeIsEnclosingType())).accept {

	provider = "Grails Constraints DSL"  // this value will appear in content assist

	// for each non-static property, there are numerous constraints "methods" that are available
	// define them all here
	for (prop in props) {
		if (prop.isStatic()) {
			continue
		}
		if (prop.type == ClassHelper.STRING_TYPE) {
			method isStatic: true, name: prop.name, params: [blank:Boolean], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [creditCard:Boolean], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [email:Boolean], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [url:Boolean], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [matches:String], useNamedArgs:true
		} else if (prop.type.name == Date.name) {
			method isStatic: true, name: prop.name, params: [max:Date], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [min:Date], useNamedArgs:true
		} else if (ClassHelper.isNumberType(prop.type)) {
			method isStatic: true, name: prop.name, params: [max:Number], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [min:Number], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [scale:Number], useNamedArgs:true
		} else if (prop.type.implementsInterface(ClassHelper.LIST_TYPE)) {
			method isStatic: true, name: prop.name, params: [maxSize:Number], useNamedArgs:true
			method isStatic: true, name: prop.name, params: [minSize:Number], useNamedArgs:true
		}
		method isStatic: true, name: prop.name, params: [unique:Boolean], useNamedArgs:true
		method isStatic: true, name: prop.name, params: [size:Integer], useNamedArgs:true
		method isStatic: true, name: prop.name, params: [notEqual:Object], useNamedArgs:true
		method isStatic: true, name: prop.name, params: [nullable:Boolean], useNamedArgs:true
		method isStatic: true, name: prop.name, params: [range:Range], useNamedArgs:true
		method isStatic: true, name: prop.name, params: [inList:List], useNamedArgs:true
	}
}

If you copy the DSLD script above and add it to a DSLD file in your Grails project, the constraints language will be taught to STS. For example, in the following simple domain class, you get this for content assist inside of the constraints block:
Using the constraints DSL

The above script can be tweaked to add custom documentation.

I use Groovy, but I don't create my own DSLs. Why should I care about DSLDs?

Even though the majority of Groovy and Grails users do not implement their own DSLs, they consume DSLs (in Grails, Gaelyk, through builders, etc.). So, even though most STS users won't be creating their own DSLDs, they will be benefiting from DSLDs created by others. We will be working hard with library and DSL developers in order to create common DSLDs for different pieces of the Groovy ecosystem.

You can expect to see a significant increase in support for the popular Groovy-based frameworks in upcoming versions of Groovy-Eclipse.

The current state of DSLD

The core implementation of the DSLD language is now available for use, but we will be tweaking it as we learn more about what users require and the kinds of DSLs that they want to support. We will be implementing more pointcuts, expanding on the documentation, and working to ship some standard DSLDs with Groovy-Eclipse itself.

Please try out some of the DSLDs introduced here or on the wiki and give us feedback on this blog post, at our issue tracker, or on the Groovy-Eclipse mailing list.

Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • Twitter
 

19 responses


  1. Nice feature. But at the same time I'm disappointed that you've created yet another DSL descriptor, when there has been one around for more than a year: http://confluence.jetbrains.net/display/GRVY/Scripting IDE for DSL awareness
    Any reason you guys can't cooperate on this?


  2. Nice feature! But at the same time I'm disappointed that you've created yet another DLS descriptor when there has been one around for more than a year: http://bit.ly/mIuc0l
    Any reason you guys can't cooperate on this?


  3. Sorry for the double post, I took a few minutes before they were visible. I fixed the url in the second post.


  4. Hi Ronny,

    We had originally hoped to make DSLD compatible with IntelliJ's GDSL. In fact, the first prototype of the language looked very similar. However, we abandoned that syntax for several reasons:

    1. Most importantly, GDSL leaks some IntelliJ-specific API from its script. In the link you provide, have a look at the section entitled "Describing GroovyDSL internal language in its own terms". Notice all the references to internal IntelliJ classes.
    2. The GDSL language is not flexible enough for our requirements. For example, there does not appear to be any way of combining GDSL contexts in the same way DSLD can combine pointcuts. In the Grails Constraint example above, we need to define a very specific location where the extra method contributions can apply (i.e., inside of a closure that is assigned to a static field named constraints in a domain class). There is no clear way that I can see this being expressed in GDSL.
    3. There is no mechanism for adding doc comments and hovers for GDSL, which we consider an integral part of the solution.

    I do think that GDSL is an impressive DSL, and I do see the benefit of having a single DSL that would have worked on both platforms, but doing so would have so severely restricted what we could have done with our own implementation that Groovy-Eclipse users would not have benefited much if we had gone this route.


  5. Thanks for the reply Andrew.

    Have you discussed these shortcomings with JetBrains, and had a dialog on what features are needed and asked if you could agree on something?
    Having a separate language for each IDE is, obviously, very counter productive.
    Are other IDE vendors free to implement DSLD?


  6. An answer to your second question first: Yes, other IDE vendors are free to implement their own version of DSLD. Groovy-Eclipse is open source (primarily ASL) and that includes the DSLD implementation. More than that, the DSLD implementation relies on the Groovy AST, and does not expose Groovy-Eclipse APIs [1]. This should facilitate other IDEs providing their own implementation.

    Now, your first question: We have not yet spoken directly with the IntelliJ team. We originally considered doing so. And indeed, we thought we would keep them in the loop with our progress, but the initial prototypes of DSLD were rough and our requirements weren't well defined. We felt that we couldn't have a productive conversation with them until we had a clear idea of what we needed to do. From a technical stance, we had two choices:

    1. Try to maintain consistency between our syntax and GDSL's. But trying to replicate internal IntelliJ APIs using Groovy AST APIs would likely make our implementation more complicated, limit our feature set, and extend the time it would take to develop.
    2. Focus on implementing the best product that we can that meets our requirements and works with our existing architecture. And hope that there is some way of reconciling the two languages later.

    We chose the second route. By the time our requirements had crystallized, DSLD syntax had drifted far enough from GDSL that maintaining consistency while still supporting our requirements would be non-trivial. I would like to have a chat with the IntelliJ developers at the upcoming Groovy conference in Copenhagen.

    [1] There is one pointcut, the "nature" pointcut, that does expose some bit of Eclipse internals. That pointcut matches on project natures. We are considering providing a different implementation of it so that the Eclipse-specific nature string is not required.


  7. Thanks again Andrew, long and detailed answer :)
    Hope JetBrains will adopt the new DSLD. Will be great when Groovy SDK, Grails, Gpars etc all comes bundled with DSLD.
    See you at Gr8conf!


  8. Hi, Andrew,

    I'm one of the GDSL creators and I must say I'm also a bit disappointed to see yet another language that does the same thing as our GDSL. I understand your points about IntelliJ API dependence and context description inflexibility. In fact, I also think these are weak points and they must be improved, but they are there for a reason. So I'm all for a unified approach, and I think we need some discussion here. Unfortunately I'm not coming to any Groovy conference in the near future, so I guess the discussion should happen online. Either here or you can email me if you want :)

    Peter


  9. Hi Peter,

    It will be good to have a conversation about this out in the open, but I'm not sure if commenting on this blog is the right place. How about I create a page on the codehaus wiki where we can discuss this further?

    It's too bad that you will not be able to make it to the conference. Will anyone else from the IntelliJ team be attending?


  10. Hi Andrew,

    My plans have suddenly changed (not without your influence), and I'll attend the conference, and we'll talk there :) Anyway, a public page won't hurt.

    Peter


  11. That's great. I'll see you there and we can plan on having a face to face discussion about this. I also cobbled together a wiki page with a few empty headings where we can start the discussion. Feel free to add to this, reorganize, etc.

    http://docs.codehaus.org/display/GROOVY/A unified approach to DSL support in IDEs


  12. Hi Andrew,

    Would not it be even more effective to infer the extension methods, properties and whatsoever from the ExpandoMetaClass of the instance in scope, rather than a custom DSL descriptor?

    Regards,
    Mitko


  13. Hi Mitko,

    The problem is that there is no running program and hence no EMC instance to inspect. DSLDs must work at edit time and hence cannot do this. Furthermore, using an EMC is only one way to dynamically add methods and properties. Using AST transforms, overriding methodMissing and getProperty are two other ways. There is no way we can discover all this information using static analysis alone.


  14. Hi Andrew,

    do you mean that it is not possible to infer the EMC extensions or that it is not possible to infer extensions for those 3 ways of adding extensions?

    I would be of great help if at least the EMC extensions were automatically infered (perhaps with combined with a feature request for groovy if something is not possible). Writing extensions and DSLDs for those extensions is a fair duplication of code, if not somehow automated.

    Are there any examples for DSLDs that provide type inference for missing methods and properites?


  15. The general case of statically discovering dynamic changes to the EMC is unsolvable. That being said, it is theoretically possible to discover some EMC changes statically while editing. But, doing so may be very processor intensive and prohibitively expensive to calculate in the background while editing.

    However, there is a research project that I am aware of that is looking at the viability of using dynamic analysis of JUnit tests to feed type inferencing inside of Groovy-Eclipse. There are some issues when attempting to do this, but this might be a viable option.


  16. Hi Andrew,
    Is there a way to import/include one DSLD into another? I'm doing something like SwingBuilder, but modularized so that components register their own factories. How do I modularize the DSLD since the currentType (the builder) is always the same?


  17. At this point, there is no way to reference other scripts from inside of a DSLD. They are meant to be standalone. As DSLDs become longer and more complex, they become harder to maintain. At some point you could start thinking about creating an Eclipse plugin for your DSL builder which gives you more fine-grained control over how type inferencing works.


  18. Hi,

    Is it possible to use third-part library in dsld scripts? If so how?


  19. @Erdam, not easily. DSLD scripts use the classpath of the running Eclipse, not the app that it is applying to. And since Eclipse uses OSGi, the jar needs to be on the path of the org.codehaus.groovy.eclipse.dsl bundle.

    What exactly are you trying to do?

3 trackbacks

Leave a Reply