Meta Model

Implementations define the featureset

Metaprogramming is hard. One of the reasons is that there are a lot of different apis each with their own philosophy, design, quirks limitations and pitfalls. This leads to a high barrier of entry and a lot of cognitive complexity. There are some projects implemented with technology A that should use B, but are unable to switch. This project aims to provide one api for Metaprogramming. At the center of it is the meta model api that represents the source code.

The project is separated into multiple maven modules. There is a core module that contains the meta model api and functionality build on top like rendering. And there are multiple modules that implement the api defined in the core module for different technologies. There is one implementation module for reflection and one for annotation processing for example.

The meta model api is consumed by two clients. There is the user of a specific Metaprogramming api like reflection. And features for every meta programming type like rendering.

When using a specific type of Metaprogramming the api needs to:

be feature complete
be discoverable
have a low barrier of entry

Features build on top of this api like rendering need

only a subset of features
consistency in the api

All features of the reflection api and the annotation processing api for example should be supported. They currently have similar featuresets, but not the same. The difference will become grater with Project Babylon.

This post explores how different implementations of the same api can have different featuresets.

One Api

Provide all the functionality in the core api. With implementations supporting the subset they can.

core

@interface Supported {

   Implementation[] value();
}

enum Implementation {

   REFLECTION,
   ANNOTATION_PROCESSING
}

public interface Method {

   @Supported({Implementation.ANNOTATION_PROCESSING, Implementation.REFLECTION})
   String getName();

   @Supported(Implementation.REFLECTION)
   Optional<FuncOp> getCodeModel();
}

implementation(reflection)

class ReflectionMethodImpl implements Method {

   private final java.lang.reflect.Method method;

   public ReflectionMethodImpl(java.lang.reflect.Method method) {

      this.method = method;
   }

   @Override
   public String getName() {

      return method.getName();
   }

   @Override
   public Optional<FuncOp> getCodeModel() {

      return method.getCodeModel();
   }
}

implementation(Annotation Processing)

class AnnotationProcessingMethodImpl implements Method {

   private final ExecutableElement executableElement;

   public AnnotationProcessingMethodImpl(ExecutableElement executableElement) {
      this.executableElement = executableElement;
   }

   @Override
   public String getName() {

      return executableElement.getSimpleName().toString();
   }

   @Override
   public Optional<FuncOp> getCodeModel() {

      throw new NotImplementedException();
   }
}

This has a low barrier of entry, because its just like most apis.
When many different apis are supported, each having its own set of functionality, it can become annoying to find the right methods to call
@Supported is a circular dependency. with each implementation the core has to be updated. Making it hard to write 3. Party implementations
There is a risk of concepts leaking from one implementation over the core-api into other implementations
easy to call the wrong methods

Inheritance

The core api contains only functionality every implementation can use. Implementations extend the model to provide additional functionality.

core

interface Method {

   String getName();
}

interface Class<METHOD extends Method> {

   List<METHOD> getMethods();
}

implementation(reflection)

interface ReflectionMethod extends Method {

   Optional<FuncOp> getCodeModel();
}

implementation(Annotation Processing)

interface AnnotationProcessingMethod extends Method {}

hard to make mistakes when using this api
low barrier of entry
good separation of concerns

Breaks all clients with every new language feature. Class for example needs the type of methods it returns. And when new core functionality gets added there maybe a need for an additional generic-parameter breaking every implementation and any code dependent on them.

interface NewFeature{}

interface Class<METHOD extends Method, NEW_FEATURE extends NewFeature> {

   List<METHOD> getMethods();

   NEW_FEATURE getNewFeature();
}
static {
   //adding a new generic would break all clients
   //they would go from
   Class<Method> original = null;
   //to
   Class<Method, NewFeature> newMethod = null;
}

Static Getter

reflection

public static String getName(Nameable nameable) {

}

implementation

static {

   Method method = null;
   String name = getName(method);
}

This is the simples possible solution where implementations define the featureset. Each implementation has their own static getter. The discoverability is poor. The caller has to know that method extends Nameable to know that getName() can be called.

For this Object → get the name should be preferred over get the name → for this Object Its more "natural this way" and generally provides better discoverability for example via IDE autocomplete.

Static Method for Query

core

public interface Query<TO_QUERY, RESULT> {}

public static  <TO_QUERY, RESULT> RESULT getOrThrow(TO_QUERY toQuery, Query<TO_QUERY, RESULT> query) {

}

implementation

public static Query<Nameable, String> name() {

}

usage

static {

    Method method = null;
    String s = getOrThrow(method, name());
}

Here the direction is turned to For this Object → get the name, but the discoverability can still be improved.

A Query can be a Function

core

public interface Query<TO_QUERY, RESULT> extends Function<TO_QUERY, RESULT> {

   @Override
   RESULT apply(TO_QUERY toQuery);
}

implementation

public static Query<Nameable, String> name() {

}

usage

static {

   Method method = null;
   String name = name().apply(method);
}

A Query is functionality equivalent to a Function. If the Query is the "active" part its back to get the name → for this Object

Duplicate Model

implementation(reflection)

public static ReflectionNameable query(Nameable nameable) {

}

public static ReflectionMethod query(Method method) {

}

public interface ReflectionNameable {
   String getName();
}

public interface ReflectionMethod extends ReflectionNameable {

   Return getReturn();
}

usage

static {

   Method method = null;
   String name = query(method).getName();
}

We can just duplicate the model or subsets of it and use static factory methods to map from one to the other. This offers the best possible discoverability. The only downside is that implementation is a lot of work.

Companions

core

public interface Executable<T extends ExecutableQuery> {
}

public interface Method<T extends MethodQuery> extends Executable<T> {
}

public interface ExecutableQuery {
}

public interface MethodQuery extends ExecutableQuery {

   void getName();
}

implementation(reflection)

public interface ReflectionExecutableQuery extends ExecutableQuery {
}

public interface ReflectionMethodQuery extends ReflectionExecutableQuery,
                                               MethodQuery {
}

When moving the methods to a companion object the model works perfectly. Problems occur when working with a nonspecific model.

public interface Converter {

   static <T> ExecutableConverter convert(Executable<T extends ExecutableQuery> executable) {
      //implementation
   }
}
public interface ExecutableConverter<T> {

   <M extends ExecutableQuery> Method<M> toMethod();
}

For a Executable<ReflectionExecutableQuery> Method<ReflectionMethodQuery> can not be returned.

Generic

core

public interface Query<TO_QUERY, RESULT> {

   static <TO_QUERY extends T, T> Queryable<T> query(TO_QUERY toQuery) {
   }
}

public interface Queryable<TO_QUERY> {

   <RESULT> RESULT getOrThrow(Query<TO_QUERY, RESULT> query);

   <RESULT> Optional<RESULT> get(Query<TO_QUERY, RESULT> query);
}

implementation

public static Query<Nameable, String> name() {}

usage

static {

   Method method = null;

   //compiles
   Queryable<Nameable> query = Query.query(method);
   query.get(name());

   //compiles
   Optional<String> s = Query.<Method, Nameable>query(method).get(name());

   //doesn't compile
   Optional<String> s1 = Query.query(method).get(name());
}

Here Query has a static factory with generics that give flexibility for supertypes. So could a Query for Nameable be applied to a Method, because Method extends Nameable. The Query call has two parts. First the Object to query gets wrapped in a Queryable which can be queried when the types match. This works but, but target type inference is limited for chain methods calls. I am not aware of any planed jdk enhancements this for this behavior. In this state the api is annoying and unintuitive to use.

Conclusion

The requirements of the two clients are so different that two apis are beneficial.

The preferred api for specific Metaprogramming implementations like reflection is "Duplicate Model". It provides the best discoverability together with "Generic" and has the lowest barrier of entry. This can not be used for common features as there are simply no methods to call in the core module.

For them, I landed on a variant of "Static Method for Query" just with an SPI.

core

public record Operation<TYPE, RESULT>(String name) {}

public interface Operations {

   static Operation<Nameable, String> NAMEABLE_NAME = new Operation<>("nameable.name");
}

public interface ProviderSpi {

   <RESULT, TYPE extends ImplementationDefined> Response<RESULT> request(TYPE instance, Operation<TYPE, RESULT> operation);
}

public sealed interface Response<T> {

   /**
    * Equivalent to Optional.empty to prevent to much nesting
    */
   final class Empty<T> implements Response<T> {}

   final class Unsupported<T> implements Response<T> {}

   record Result<T>(T value) implements Response<T> {}
}

usage

static {

   Method method = null;
   String name = requestOrThrow(method, NAMEABLE_NAME);
}