In this page you will learn about the Scala IDE’s architecture. First, an overview of the different existing modules is provided. Then, a few specific aspects of the Scala IDE’s architecture and its components are discussed.
The Scala IDE project is composed of several modules. Here is a generic description of what each module contains:
- org.scala-ide.build: Contains the scripts for building the Scala IDE from the command line.
- org.scala-ide.build-toolchain: Contains the script for retrieving and repackaging dependencies that are needed by the Scala IDE.
- org.scala-ide.sbt.full.library: Packages the Sbt incremental compiler into one OSGi bundle.
- org.scala-ide.sbt.compiler.interface Packages the Sbt compiler interface into an OSGi bundle
- org.scala-ide.sdt.aspects: Contains the AspectJ classes used to weave into Eclipse and hook in JDT internals.
- org.scala-ide.sdt.core: Contains the Scala IDE plug-in’s source code.
- org.scala-ide.sdt.core.tests: Contains the functional tests used to exercise the Scala IDE in headless mode (with no User Interface).
- org.scala-ide.sdt.debug: Contains the Scala IDE debugger plug-in’s source code.
- org.scala-ide.sdt.debug.tests: Contains the functional tests used to exercise the Scala IDE debugger in headless mode (with no User Interface).
- org.scala-ide.sdt.spy: A Scala Spy view, used to display position, tree and type information under the cursor.
- org.scala-ide.sdt.feature: Packages the org.scala-ide.sdt.core and org.scala-ide.sdt.debug project binaries into the “Scala IDE for Eclipse” component, which is then made available through the Scala IDE update site.
- org.scala-ide.sdt.source.feature: Packages both the org.scala-ide.sdt.core and org.scala-ide.sdt.aspects project sources into the “Scala IDE for Eclipse Source Feature” component, which is then made available through the Scala IDE update site.
- org.scala-ide.sdt.weaving.feature: Packages the org.scala-ide.sdt.aspects project binaries into the “JDT Weaving for Scala” component, which is included within the “Scala IDE for Eclipse” component.
- org.scala-ide.sdt.update-site: Contains the result of a successful compilation of the Scala IDE project, packaged in the form expected by the Eclipse Update Manager.
The Build system
The Scala IDE build system is based on Maven and Tycho, but it has a few particularities. In most cases running the build-all.sh script should be sufficient, but in case you need to make changes or debug a failed nightly, the following information may prove useful.
Tycho brings Eclipse specific dependency management to Maven, essentially resolving Require-Bundle or Import-Package entries in MANIFEST.MF files. To this end it scans manifests early on in the build cycle and uses P2 repositories to locate the best match for each dependency. In other words, it builds a so-called target-platform, containing all the required bundles (jar files).
In the real world not all dependencies will be available in P2 repositories. Tycho considers plain Maven (pom) dependencies as well, as long as the jar files have a MANIFEST.MF file. This is the case for Scala: the standard library, compiler and Scala modules (from 2.11 onwards) have correct MANIFEST.MF files, and the Scala IDE build is using them directly in its build.
In the worst case, the required dependency is only available as a plain jar, but no MANIFEST file is provided. The solution is to re-package them using the maven bundle plugin who generates a manifest. This is what the org.scala-ide.sbt.full.library and org.scala-ide.sbt.compiler.interface projects are doing for the Sbt incremental compiler.
Tycho requires pom-first dependencies to be available in a maven repository during the initialize maven lifecycle. This means that repackaged dependencies have to be built and published in a separate maven run. Listing them in the same parent pom as separate modules won’t cut it. This is the reason why we need the toolchain build.
The set-featues step
Currently we cross-build against Scala 2.10 and 2.11. Due to modularization, the Scala bundles we have to ship differ between the two versions, leading to different features.xml files in the two cases. Unfortunately, due to Tycho building the target platform so early in the build lifecycle, we cannot produce these files in the same maven run, or even the same build. This is handled by using the set-features profile in the root pom.xml, and called by build.sh before handing it over to the “main” build.
Scala IDE & Eclipse JDT
The Scala IDE uses the Eclipse Java Development Tools (JDT) weaving in order to inherit as much as possible of the existing Eclipse Java functionality. AspectJ and the weaving service of Eclipse (in fact, its OSGi container, called Equinox) allow us to intercept method calls in the JDT, and provide Scala-specific implementations for things like the source file editor or the indexer.
By hooking into the JDT, and providing specific implementations of JDT interfaces, Scala entities (classes, types, methods, etc) are seen by the JDT as Java elements, meaning:
- Java code completion “sees” Scala classes.
- The Java presentation compiler “sees” errors when using Scala classes.
- JDT tools, such as the JUnit runner, just ‘work’.
- Lots of UI elements can be reused (the Outline View, Package Explorer, etc).
Of course, Scala entities can not be faithfully represented in Java terms. Some Scala features will be therefore only partially supported when working in mixed Java-Scala projects (for instance, type members are represented as normal, value fields).
This will hopefully change in the near future, as we plan to create our own models for representing Scala entities and hook that into JDT (rather than the current solution, where we are using Java models to expose Scala entities).
Although not intended to be extended with other languages, the JDT offers extension points for some functionality. Whenever possible, we prefer to use the regular, supported mechanism for extension, instead of weaving. This is the case for both code completion and hyperlinking.
JDT Weaving and Java integration
All Scala aspect code is found in org.scala-ide.sdt.aspects project. This module defines:
- pointcuts and aspects to intercept JDT method calls,
- Java interfaces and classes that are used by these aspects. The main IDE module, org.scala-ide.sdt.core implements or inherits these classes. This way, there is a clean separation between the hooks, and the actual implementations.
The other side of this coin is the actual Scala-specific implementation in org.scala-ide.sdt.core. The package scala.tools.eclipse.javaelements contains Scala subclasses of Java element classes, such as ScalaClassElement or ScalaDefElement.
Although the JDT defines interfaces for these elements, we inherit from their internal implementations. This makes our code prone to breakage in newer releases of the JDT, but gives us much more functionality. In some cases it is even impossible to do otherwise, as the JDT code itself casts such interfaces to their concrete, internal, implementations.
As mentioned in the previous section, we plan to create our own models for representing Scala entities. That will help us better representing Scala entities in the editor and it will mitigate incompatibilities with newer version of the JDT.
Scala IDE debugger
The Scala IDE debugger is using the standard Java debugging chain (JVMDI, JDWP and JDI) to provide a Scala specific debugger. The elements are displayed the Scala way, the names are decoded, and the actions are behaving the way a Scala developer would expect.
The Scala Structure Builder
The workhorse of Java-Scala integration is the ScalaStructureBuilder class. This class translates Scala sources (using the Abstract Syntax Trees - AST) to the JDT model elements (using the Scala specific subclasses, of course), and it is fully responsible for the way the Eclipse Java compiler (and related JDT tools or UI elements) sees Scala code.
This approach has some important limitation with respect to correct interoperation of mixed Scala/Java code, and it is the source of several issues in the Scala IDE. One major point of pain is that the Scala Structure Builder needs to faithfully emulate the Scala compiler to provide correct interoperation between Scala and Java code. Doing this in general is:
- difficult, as we are somewhat trying to re-engineer what the compiler already does when it produces bytecode, and
- fragile, as the produced bytecode may vary from one Scala version to another.
In the future we would like to explore other roads. For instance, we have been thinking about using the binaries (instead of the sources), so that we would not need to duplicate the Scala compiler backend’s logic. Though, doing this would force users working on mixed Scala/Java projects to enable continuous build (to make sure that the class binaries are actually produced), which may not be acceptable.
The Scala Presentation Compiler
In order to provide semantic actions, the IDE needs to understand the edited Scala code. That means parsing and type-checking. The Scala Presentation Compiler is an asynchronous front-end compiler for Scala, part of the standard Scala compiler.
Refactoring is delegated to the scala-refactoring library written by Mirko Stocker.