My Forge Experience pt1

In this post i’ll share my experience with JBoss Forge 1.x[1], i will cover Forge 2.x[2] in pt2.

So the main objective of this entry is to create a forge plugin that get useful information from OSGi[3][4][5] projects.

But what is Forge? and what is OSGi?

to be straightforward forge is a plugin based java command line tool(in forge 2.x this definition may change bit) based on CDI[6] providing a rich API to create and/or manipulate java projects which is the main purpose of forge but not limited to it.

OSGi is the de facto standard for building modular, dynamic, service based java applications.

One of the benefits of Forge is to create, configure and add features to projecs such as JPA, REST, JSF functionality and so on, in this post we are NOT going to add any feature neither configure projects, instead we are going to inspect and extract data from existing projects. To be more exact we are going to build a Forge plugin to get information from OSGi based projets, such as the one from this great paper introducing OSGi: http://homepages.dcc.ufmg.br/~mtov/pub/2008_sen.pdf.

Before we ge our hands dirty here is a video showing the result project of this post: http://youtu.be/rS-6LuMWPHI and
the source code is available at github: https://github.com/rmpestano/intrabundle

Forge Configuration

First thing to do is to start forge, you’ll need to download a zipped file[12], unconpress and execute forge file (windows, linux and osx compatible).
Optional step is to add forge to your system path, in linux you can add the following line to ~.profile file:
export PATH=$PATH:/home/rmpestano/projetos/forge/dist/forge-distribution-1.4.3.Final/bin(on windows just add …..forge-distribution-1.4.3.Final/bin to path environment variable). For detailed information see[7]

OBS: For debugging purposes you can also add to ~.profile: export FORGE_OPTS=”-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000″(on windows create FORGE_OPTS enviromnent variable) so you can attach a remote debugger to your plugin.

Creating the plugin project

After that we are going to create our plugin project, forge will help us on that:
Open a terminal and start forge by typing ‘forge'(if you didnt added forge to your environment cd into forge-distribution/bin and execute the command) as image 1:

pic01

image 1

now change dir to a folder of your choice and type new-project –named intrabundle –topLevelPackage br.ufrgs.rmpestano.intrabundle
to create the project.

Next lets create our forge plugin, first we need to setup plugin dependencies, type plugins setup and accept the defaults by clicking ENTER.

OBS: you can cd to pom.xml and type ‘ls’ to confirm that dependencies were added, see image 2:

pic03

image 2

After setting up plugin the plugins new-plugin command becomes available and here a (big) parenteses: How that command showed up now?
One great thing of Forge is that it is build up by forge plugins so we gain for free a lot of (nice)plugins examples so looking at
org.jboss.forge.dev.PluginsPlugin.java we can see how that “magic” works, basically it is annotated with ‘@RequiresFacet(ForgeAPIFacet.class)’ annotation which tells forge that this plugin’s commands(except the @SetupCommand) can only be executed in certain context and who dictates this context is the required facet, in this case ForgeAPIFacet.java. The context is ‘alive’ when facet isInstalled() method returns true. Take a look at [8] for more information about facets.

Closing our big parenteses, lets execute plugins new-plugin –named OSGiPlugin –alias osgi

Now you can see OSGiPlugin.java file was created by forge with some basic commands. Also OSGiPluginTest.java was created so you can test your plugins without starting forge and installing the plugin. To do that forge leverages Arquillian framework [9] the de facto framework for testing JavaEE applications. To run the generated tests(via IDE) just right click in OSGiPluginTest and Run as JUnit test or (via forge) run ‘build’ command on project, see [11] for more detais on testing plugins.
Lets take a look at OSGiPlugin.java:

package br.ufrgs.rmpestano.intrabundle;
//imports ommited
@Alias("osgi")
public class OSGiPlugin implements Plugin
{
   @Inject
   private ShellPrompt prompt;

   @DefaultCommand
   public void defaultCommand(@PipeIn String in, PipeOut out)
   {
      out.println("Executed default command.");
   }

   @Command
   public void command(@PipeIn String in, PipeOut out, @Option String... args)
   {
      if (args == null)
         out.println("Executed named command without args.");
      else
         out.println("Executed named command with args: " + Arrays.asList(args));
   }

   @Command
   public void prompt(@PipeIn String in, PipeOut out)
   {
      if (prompt.promptBoolean("Do you like writing Forge plugins?"))
         out.println("I am happy.");
      else
         out.println("I am sad.");
   }
}

It has three commands denoted by @Command annotation, the name of the command is the name of the method(you can provide the name via ‘value’ attribute in the command annotation). Every command is prefixed by plugin alias, @Alias anotation at class level, plus command name (in our case @Alias(“osgi”)). The default command has the same name as plugin alias.

 Installing the Plugin

To install our plugin and start executing commands type forge source-plugin “project location” as image 3.1 and image 3.2

img04.0

image 3.1

img04

image 3.2

now you can execute the commands by typing ‘osgi command’ (use tab for autocompletion).

If your project is available at a git project repository such as github you can install your plugin directly from it using forge git plugin command, for our plugin you should use: forge git-plugin git://github.com/rmpestano/intrabundle.git. See image 4:

pic02

image 4 

that one was easy, but as you can see you can execute the commands regardless the location or project you are and our idea is to execute commands on top of OSGi based projects. To do that we are going to create our plugin facet(the same idea behind ‘plugins new-plugin’ command we talked earlier)

The OSGi Facet

To restrict our plugin to OSGi projects we need to specify what is the prerequisite that a project must satisfy to be considered OSGi based.

One thing that differ OSGi projects from others is the presence of OSGi metadata in MANIFEST.MF file. So this is what OSGi Facet will look for, also we will consider that OSGi projects have a parent folder and inside it has its modules(a.k.a bundles) so the algorithm of OSGiFacet will go down two levels of directories after META-INF folder, if it find META-INF folders it will try to find MANIFEST file inside, if it succeeds it will read the file looking for OSGi metadata, if it finds any the OSGiFacet is satisfied and we will be able to execute our plugin commands, here is the code:

package br.ufrgs.rmpestano.intrabundle;

import org.jboss.forge.project.facets.BaseFacet;
import org.jboss.forge.resources.DirectoryResource;
import org.jboss.forge.resources.Resource;
import org.jboss.forge.resources.ResourceFilter;

import javax.inject.Singleton;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.ArrayList;
import java.util.List;

@Singleton
public class OSGiFacet extends BaseFacet {

    private int metaInfSearchLevel = 1;//defines how much directory levels to go down looking for OSGi metadata(manifest file)

    @Override
    public boolean install() {

        /**
         * we are not going to install OSGi projects,
         * just analyse existing ones
         *
         */
        return isInstalled();
    }

    @Override
    public boolean isInstalled() {
        return isOSGiProject(project.getProjectRoot());

    }

    /**
     * search OSGi metadata looking for META-INF directory with manisfest.mf file
     * containing the 'bundle' word
     *
     * @param directoryResource
     * @return
     */
    public boolean isOSGiProject(DirectoryResource directoryResource) {
        List<Resource<?>> metaInfList = new ArrayList<Resource<?>>();

        this.getMetaInfDirectories(directoryResource, metaInfList, 0);

        if (metaInfList.isEmpty()) {
            return false;
        }
        for (Resource<?> metaInf : metaInfList) {
            if (isOSGiModule(metaInf)) {
                return true;
            }
        }
        return false;
    }

    /**
     * gather META-INF directories by looking
     * for each @parent directory get its meta-inf directory
     * until metaInfSearchLevel is reached
     *
     * @param parent
     * @param resourcesFound
     * @param currentLevel
     */
    public void getMetaInfDirectories(DirectoryResource parent, List<Resource<?>> resourcesFound, int currentLevel) {
        if (currentLevel >= metaInfSearchLevel) {
            return;
        }
        for (Resource<?> r : parent.listResources()) {
            if (r instanceof DirectoryResource) {
                resourcesFound.addAll(r.listResources(new ResourceFilter() {
                    @Override
                    public boolean accept(Resource<?> resource) {
                        return resource.getName().equalsIgnoreCase("meta-inf");
                    }
                }));
                getMetaInfDirectories((DirectoryResource) r, resourcesFound, ++currentLevel);
            }
        }

    }

    private boolean isOSGiModule(Resource<?> metaInf) {
        Resource<?> manifest = metaInf.getChild("MANIFEST.MF");
        if (!manifest.exists()) {
            return false;
        }
        RandomAccessFile randomAccessFile;
        try {
            File f = new File(manifest.getFullyQualifiedName());
            randomAccessFile = new RandomAccessFile(f, "r");
            return hasOsgiConfig(randomAccessFile);
        } catch (Exception e) {
            e.printStackTrace();
            return false;
        }
    }

    private boolean hasOsgiConfig(RandomAccessFile aFile) throws IOException {
        String line;
        while ((line = aFile.readLine()) != null) {
            if (line.contains("Bundle")) {
                return true;
            }
        }
        return false;

    }

    public int getMetaInfSearchLevel() {
        return metaInfSearchLevel;
    }

    public void setMetaInfSearchLevel(int metaInfSearchLevel) {
        this.metaInfSearchLevel = metaInfSearchLevel;
    }
}

now add requiresFacet to our OSGiPlugin and install it again

@Alias("osgi")
@RequiresFacet(OSGiFacet.class)
public class OSGiPlugin implements Plugin

now you can only execute the commands in OSGi projects, the ones which have a subfolder with meta-inf folder containing a MANIFEST.MF file with osgi metadata. You can find an example OSGi project in [10] it’s from the paper ‘A gentle Introduction to OSGi’[13].

Before testing the plugin there is one problem, one limitation of Forge1.x is that it needs a pom.xml file in project root to work, in other words it was made for maven projects. Most OSGi projects use eclipse bnd tools plugin, some use maven plus maven bundle plugin but we are going to focus on non maven ones. To surpass this limitation we are going to add a minimal pom.xml in our OSGi projects and to do that we are going to create a forge Project Locator.

OSGi Project Locator

A project locator is responsible for finding and creating forge projects, they are called by org.jboss.forge.project.services.ProjectFactory#findProject() every time we change folder.

A forge project is an object that holds information about projects such as directory location, the facets it satisfies and so on.

So when we cd into an OSGi project our locator will create a java object representing it, also it will create a minimal pom.xml in project root to overcome forge1 limitation we talked about, here is the interface our Project will implement:

public interface OSGiProject extends Serializable{

    List<OSGiModule> getModules();
}

basically our OSGi project will hold a list of OSGi modules:

import org.jboss.forge.project.Project;
import org.jboss.forge.resources.FileResource;

import java.io.Serializable;

public interface OSGiModule extends Serializable,Project {

    Long getLinesOfCode();

    Boolean getUsesDeclarativeServices();

    FileResource<?> getManifest();

    FileResource<?> getActivator();
}

here is OSGiProjectImpl.java

import br.ufrgs.rmpestano.intrabundle.facet.OSGiFacet;
import org.jboss.forge.project.BaseProject;
import org.jboss.forge.project.Project;
import org.jboss.forge.project.Facet;
import org.jboss.forge.project.facets.FacetNotFoundException;
import org.jboss.forge.project.services.ProjectFactory;
import org.jboss.forge.resources.DirectoryResource;
import org.jboss.forge.resources.Resource;

import javax.enterprise.inject.Typed;
import java.util.ArrayList;
import java.util.List;

@Typed()
public class OSGiProjectImpl extends BaseProject implements OSGiProject,Project {
    private DirectoryResource projectRoot = null;
    private final ProjectFactory factory;
    private List<OSGiModule> modules;

    public OSGiProjectImpl() {
        factory = null;
    }

    public OSGiProjectImpl(final ProjectFactory factory, final DirectoryResource dir) {
        this.factory = factory;
        this.projectRoot = dir;
    }

    @Override
    public <F extends Facet> F getFacet(final Class type) {
        try {
            return super.getFacet(type);
        } catch (FacetNotFoundException e) {
            factory.registerSingleFacet(this, type);
            return super.getFacet(type);
        }
    }

    public List<OSGiModule> getModules() {
        if (modules == null) {
            modules = initalizeModules();
        }
        return modules;
    }

    private List<OSGiModule> initalizeModules() {
        List<OSGiModule> modulesFound = new ArrayList<>();
        OSGiFacet osgi = getFacet(OSGiFacet.class);
        List<Resource<?>> metaInfList = new ArrayList<Resource<?>>();
        osgi.getMetaInfDirectories(this.getProjectRoot(), metaInfList, 0);
        for (Resource<?> resource : metaInfList) {
            OSGiModule osGiModule = new OSGiModuleImpl(factory, (DirectoryResource) resource.getParent());
            modulesFound.add(osGiModule);
        }
        return modulesFound;
    }

    @Override
    public DirectoryResource getProjectRoot() {
        return projectRoot;
    }

    @Override
    public boolean exists() {
        return (projectRoot != null) && projectRoot.exists();
    }

    @Override
    public String toString() {
        return "OSGiProjectImpl [" + getProjectRoot() + "]";
    }

}

and OSGiModuleImpl.java

import org.jboss.forge.project.BaseProject;
import org.jboss.forge.project.Facet;
import org.jboss.forge.project.facets.FacetNotFoundException;
import org.jboss.forge.project.services.ProjectFactory;
import org.jboss.forge.resources.DirectoryResource;
import org.jboss.forge.resources.FileResource;
import org.jboss.forge.resources.Resource;

import javax.enterprise.inject.Typed;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;

@Typed()
public class OSGiModuleImpl extends BaseProject implements OSGiModule {
    private DirectoryResource projectRoot = null;
    private final ProjectFactory factory;
    private Long totalLoc;
    private Boolean usesDeclarativeServices;
    private FileResource<?> activator;
    private FileResource<?> manifest;

    public OSGiModuleImpl() {
        factory = null;
    }

    public OSGiModuleImpl(final ProjectFactory factory, final DirectoryResource dir) {
        this.factory = factory;
        this.projectRoot = dir;
    }

    @Override
    public <F extends Facet> F getFacet(final Class type) {
        try {
            return super.getFacet(type);
        } catch (FacetNotFoundException e) {
            factory.registerSingleFacet(this, type);
            return super.getFacet(type);
        }
    }

    @Override
    public DirectoryResource getProjectRoot() {
        return projectRoot;
    }

    @Override
    public boolean exists() {
        return (projectRoot != null) && projectRoot.exists();
    }

    @Override
    public String toString() {
        return getProjectRoot().toString();
    }

    private FileResource<?> findActivator() throws IOException {
        RandomAccessFile randomAccessFile;
        File f = new File(getManifest().getFullyQualifiedName());
        randomAccessFile = new RandomAccessFile(f, "r");

        String line;
        while((line = randomAccessFile.readLine()) != null){
            if (line.contains("Bundle-Activator:")) {
               break;
            }
        }
        if(line == null){
            return null;//no activator
        }
        String actvatorPath = line.trim().substring(line.indexOf("Bundle-Activator:") + 17);
        actvatorPath = actvatorPath.trim().replaceAll("\\.","/");
        if(!actvatorPath.startsWith("/")){
            actvatorPath = "/" +actvatorPath;
        }
        actvatorPath = "/src"+actvatorPath;
        Resource<?> activator = getProjectRoot().getChild(actvatorPath.concat(".java"));
        if(activator == null || !activator.exists()){
            throw new RuntimeException("Could not find activator class at "+getProjectRoot() + actvatorPath);
        }

        return (FileResource<?>) activator;

    }

    private Long countModuleLines(DirectoryResource projectRoot) {
        for (Resource<?> resource : projectRoot.listResources()) {
            if (resource instanceof FileResource<?> && resource.getName().endsWith(".java")) {
                try {
                    this.totalLoc += countFileLines((FileResource<?>) resource);
                } catch (Exception e) {
                    e.printStackTrace();
                }
            } else if (resource instanceof DirectoryResource) {
                this.totalLoc = countModuleLines((DirectoryResource) resource);
            }
        }
        return totalLoc;
    }

    private Long countFileLines(FileResource<?> resource) throws IOException {
        RandomAccessFile file = new RandomAccessFile(new File(resource.getFullyQualifiedName()), "r");
        Long total = new Long(0);
        String line;
        while ((line = file.readLine()) != null) {
            total++;
        }
        return total;
    }

    private boolean usesDeclarativeServices() {
        Resource<?> OSGiInf = getProjectRoot().getChild("OSGI-INF");
        return OSGiInf.exists() && OSGiInf.getChild("service.xml").exists();
    }

    //getters

    public Boolean getUsesDeclarativeServices() {
        if (usesDeclarativeServices == null) {
            usesDeclarativeServices = usesDeclarativeServices();
        }
        return usesDeclarativeServices;
    }

    @Override
    public FileResource<?> getActivator() {
        if (activator == null) {
            try {
                activator = findActivator();
            } catch (IOException e) {
                throw new RuntimeException("Could not find activator class");
            }
        }
        return activator;
    }

    @Override
    public FileResource<?> getManifest() {
        if (manifest == null) {
            manifest = findManifest();
        }
        return manifest;
    }

    private FileResource<?> findManifest() {
        Resource<?> metaInf = getProjectRoot().getChild("META-INF");
        if (metaInf == null || !metaInf.exists()) {
            throw new RuntimeException("OSGi project(" + getProjectRoot().getFullyQualifiedName() + ") without META-INF directory cannot be analysed by intrabundle");
        }
        Resource<?> manifest = metaInf.getChild("MANIFEST.MF");
        if (manifest == null || !manifest.exists()) {
            throw new RuntimeException("OSGi project(" + getProjectRoot().getFullyQualifiedName() + ") without MANIFEST.MF file cannot be analysed by intrabundle");
        }
        return (FileResource<?>) manifest;
    }

    public Long getLinesOfCode() {
        if (totalLoc == null) {
            totalLoc = new Long(0L);
            totalLoc = countModuleLines(getProjectRoot());
        }
        return totalLoc;
    }
}

Im not get into details of implementation here but you can see that i’m just using forge api and standard java file manipulation to get module information, such as its location, lines of code and so on.

Back to OSGi project locator, the guy that will in fact create the OSGiProject and add minimal pom.xml(the pom.xml file to be added must be in intrabundle/src/main/resources folder):

@Singleton
public class OSGiProjectLocator implements ProjectLocator {

    private final ProjectFactory factory;

    private final Instance osgiFacetInstance;

    @Inject
    Shell shell;

    @Inject
    public OSGiProjectLocator(final ProjectFactory factory, @Any final Instance osgiFacet) {
        this.factory = factory;
        this.osgiFacetInstance = osgiFacet;
    }

    @Override
    public Project createProject(DirectoryResource directoryResource) {
        OSGiFacet osgi = osgiFacetInstance.get();
        OSGiProjectImpl result = new OSGiProjectImpl(factory, directoryResource);
        osgi.setProject(result);
        /* we are not going to install OSGi projects, only inspect existing ones
        if (!osgi.isInstalled()) {
            result.installFacet(osgi);
        } else    */
        result.registerFacet(osgi);

        if (!result.hasFacet(OSGiFacet.class)) {
            return null;
        }
	//FORGE limitation of having a pom.xml in project root
        if(!directoryResource.getChild("pom.xml").exists()){
            FileResource<?> pom = (FileResource<?>) directoryResource.getChild("pom.xml");
            pom.setContents(getClass().getResourceAsStream("/pom.xml"));
        }
        shell.println(ShellColor.YELLOW,"OSGi project detected, type osgi + tab to list available commands");
        return result;
    }

    @Override
    public boolean containsProject(DirectoryResource directoryResource) {
        return osgiFacetInstance.get().isOSGiProject(directoryResource);
    }
}

So the locator will act only on projects that satisfies OSGiFacet. Forge will use it automactly cause ProjectFactory iterates over all classes implementing ProjectLocator which is the case of OSGiProjectLocator.

Now install the project again and you are ready to use our plugin in non maven based projects.

Implementing OSGiPlugin commands

As you saw in OSGiModuleImpl we already have some methods that get information from OSGi projects but how our plugin can access OSGiProject and get its modules?

As forge leverages CDI programming model we just Inject current project in the plugin with @Inject OSGiProject project. Cause we created OSGiProjectImpl via new operator in OSGiProjectLocator CDI is not aware of it so we need to produce OSGiProject so CDI can handle its injection.

We will produce it in OSGiFacet which holds current OSGi project(setted by OSGiProjectLocator#createProject), here is the producer method:

@Produces
public OSGiProject getCurrentOSGiProject() {
    return (OSGiProject) getProject();
}

now we have access to the current OSGiProject and its modules via CDI Injection, here is the OSGiPlugin commands implementation:

package br.ufrgs.rmpestano.intrabundle.plugin;

import br.ufrgs.rmpestano.intrabundle.facet.OSGiFacet;
import br.ufrgs.rmpestano.intrabundle.i18n.ResourceBundle;
import br.ufrgs.rmpestano.intrabundle.model.OSGiModule;
import br.ufrgs.rmpestano.intrabundle.model.OSGiProject;
import org.jboss.forge.shell.ShellColor;
import org.jboss.forge.shell.ShellPrompt;
import org.jboss.forge.shell.plugins.*;

import javax.enterprise.inject.Instance;
import javax.inject.Inject;
import java.util.List;

@Alias("osgi")
@RequiresFacet(OSGiFacet.class)
public class OsgiPlugin implements Plugin {

    @Inject
    private ShellPrompt prompt;

    @Inject
    OSGiProject project;

    @Inject
    @Current
    Instance<ResourceBundle> resourceBundle;

    @DefaultCommand
    public void defaultCommand(@PipeIn String in, PipeOut out) {
        out.println(ShellColor.YELLOW, resourceBundle.get().getString("osgi.defaultCommand"));
    }

    @Command(value = "countBundles")
    public void countBundles(@PipeIn String in, PipeOut out) {
        out.println("Total number of bundles:" + getModules().size());
    }

    @Command(value = "listBundles")
    public void listBundles(@PipeIn String in, PipeOut out) {
        for (int i = 0; i < getModules().size(); i++) {
            out.println("bundle(" + i + "):" + getModules().get(i).getProjectRoot());
        }
    }

    @Command(value = "loc", help = "count lines of code of all bundles")
    public void loc(@PipeIn String in, PipeOut out) {
        long total = 0;
        for (int i = 0; i < getModules().size(); i++) {
            long loci = getModules().get(i).getLinesOfCode();
            out.println(getModules().get(i).getProjectRoot() + ":" + loci);
            total += loci;
        }
        out.println("Total lines of code:" + total);
    }

    @Command(value = "usesDeclarativeServices", help = "list modules that use declarative services")
    public void usesDeclarativeServices(@PipeIn String in, PipeOut out) {
        out.println(resourceBundle.get().getString("osgi.declarativeServices"));
        for (OSGiModule module: getModules()) {
            if(module.getUsesDeclarativeServices()){
                out.println(module.toString());
            }
        }
    }

    @Command(value = "listActivators", help = "list modules activator classes")
    public void listActivators(@PipeIn String in, PipeOut out) {
        out.println(resourceBundle.get().getString("osgi.listActivators"));
        for (OSGiModule module: getModules()) {
             out.println(module.toString()+":"+(module.getActivator() != null ? module.getActivator().getFullyQualifiedName() : resourceBundle.get().getString("osgi.no-activator")));
        }
    }

    public List getModules() {
        return project.getModules();
    }

}

Conclusion

Jboss Forge is a great tool, has great API for manipulating projects and a very nice and easy to understand architecture. We saw here a simple plugin that inspects OSGi project files and structure, your imagination is the limit for creating plugins.

I hope you enjoy.

References

[1]http://forge.jboss.org/index.html
[2]https://github.com/forge/core
[3]www.osgi.org/‎
[4]http://www.osgi.org/Technology/HowOSGi
[5]https://rpestano.wordpress.com/2013/03/14/hello-osgi/
[6]http://docs.jboss.org/weld/reference/latest/en-US/html/
[7]http://forge.jboss.org/docs/using/#content
[8]http://forge.jboss.org/docs/plugin_development/facets.html#content
[9] Arquillian.org/
[10]www.dcc.ufmg.br/~mtov/osgi_example.zip
[11]http://forge.jboss.org/docs/plugin_development/test-plugins.html#content
[12]Forge1 zip distribution

[13]http://homepages.dcc.ufmg.br/~mtov/pub/2008_sen.pdf