Metadata API

NXCALS meta-data can be retrieved using Java Client API from cern.nxcals.api.metadata package.

NXCALS Service Client API itself consist of four end points:

• SystemSpecService
• EntityService
• VariableService
• HierarchyService

Full Java documentation is available here.

In contrary to versions prior to the 0.2.0 the API of each endpoint contains similar search methods that accept arbitrary Condition built with a query builder alowing to search for one or many objects in a very flexible way. The method finding one object (findOne and findById) are cached.

Set<X> findAll(Condition<Xs>)
Optional<X> findOne(Condition<Xs>)
Optional<X> findById(long id)

Please note that you will find corresponding Condition builders with classes that names are made plural from the domain object. So Entity -> Entities, Variable -> Variables, etc

In order to build a Condition please use the following code:

Entities.suchThat().systemId().eq(1L).and().keyValues().like("%myDevice%");

Note that conditions are by default in case-sensitive mode. Meaning:

Groups.suchThat().name().eq("myName");

Will only match group named "myName".

If you wish to enable case-insensitive search please use 'caseInsensitive()' call prior to any string operation:

Groups.suchThat().name().caseInsensitive().eq("myName");

This will match every possible combination of casing of "myName", eg. "MYNAME", "MyName", "MyNaMe" etc.

Case insensitive search is also available for key-value searches:

Entities.suchThat().keyValues().caseInsensitive().like("%myDevice%myProperty%");

Conditions must return a correct number of records according to the method used. If a Condition that returns multiple rows is passed to the findOne method an exception will be thrown (IllegalArgumentException).

Important

Using case insensitive search might cause a query to return more than one value in unexpected circumstances. Be sure to call 'queryAll' method for safety, whenever opting for case-insensitive search.

System Service

Service responsible for retrieving data related to systems.

Additionally available methods (on top of findAll, findOne and findAll()):

Optional<System> findByName(String name)
Set<System> findAll()

More info can be found in Javadoc for SystemSpecService interface.

Examples

Find information about system using its name:

Java

SystemSpecService systemService = ServiceClientFactory.createSystemSpecService();

SystemSpec systemData = systemService.findByName("CMW")
        .orElseThrow(() -> new IllegalArgumentException("No such system"));

Entity Service

Service responsible for retrieving data related to Entities.

Note

For update of entities a role with WRITE permission for a given system is required. Please follow procedure described here to obtain required authorization.

In addtion to findAll, findOne and findAll(), the EntityService exposes two additional methodes for obtaining the Entity history(ies) for a given time window:

Set<Entity> findAllWithHistory(Condition<Entities> condition, long historyStartTime, long historyEndTime)
Optional<Entity> findOneWithHistory(Condition<Entities> condition, long historyStartTime, long historyEndTime)

There are some other methods for data persitance / manipulation:

Entity createEntity(long systemId, Map<String, Object> entityKey, Map<String, Object> partitionKey)
Set<Entity> updateEntities(Set<Entity> entityDataList)

More info can be found in Javadoc for EntityService interface.

Examples

Search for one entity.

Please note that extracted entity history contains only a current history:

Java

EntityService entityService = ServiceClientFactory.createEntityService();
SystemSpecService systemService = ServiceClientFactory.createSystemSpecService();

SystemSpec systemData = systemService.findOne(SystemSpecs.suchThat().name().eq("CMW"))
        .orElseThrow(() -> new IllegalArgumentException("System not found"));

Map<String, Object> keyValues = ImmutableMap.of("device", "myDevice", "property", "myProperty");
Entity entityData = entityService.findOne(
        Entities.suchThat().systemId().eq(systemData.getId()).and().keyValues().eq(systemData, keyValues))
        .orElseThrow(() -> new IllegalArgumentException("Entity not found"));

Search for an entity with its history which is present in the provided time range.

For the example below, two entity histories will be shown: first with the validity from 1970-01-01T00:00:00Z until 2019-01-01T00:00:00Z, and second with the validity from 2019-01-01T00:00:00Z onwards:

Java

SystemSpec systemData = ServiceClientFactory.createSystemSpecService()
        .findOne(SystemSpecs.suchThat().name().eq("CMW"))
        .orElseThrow(() -> new IllegalArgumentException("System not found"));

Map<String, Object> keyValues = ImmutableMap.of("device", "LHC.LUMISCAN.DATA", "property", "CrossingAngleIP1");

EntityService entityService = ServiceClientFactory.createEntityService();
Entity entities = entityService.findOneWithHistory(
        Entities.suchThat().systemName().eq(systemData.getName()).and().keyValues().eq(systemData, keyValues),
        TimeUtils.getNanosFromString("2017-10-10 14:15:00.000000000"),
        TimeUtils.getNanosFromString("2020-10-26 14:15:00.000000000")
).orElseThrow(() -> new IllegalArgumentException("Entity not found"));

entities.getEntityHistory().forEach(System.out::println);

Update entity (change its key values):

Java

SystemSpecService systemService = ServiceClientFactory.createSystemSpecService();
EntityService entityService = ServiceClientFactory.createEntityService();

SystemSpec systemData = systemService.findByName("CMW")
        .orElseThrow(() -> new IllegalArgumentException("No such system CMW"));

Map<String, Object> keyValues = ImmutableMap.of("device", "LHC.LUMISCAN.DATA", "property", "CrossingAngleIP1");
Entity entityData = entityService.findOne(Entities.suchThat()
        .systemId().eq(systemData.getId())
        .and()
        .keyValues().eq(systemData, keyValues))
        .orElseThrow(() -> new IllegalArgumentException("Entity not found"));

// Change entity to set new keyValues
Map<String, Object> newKeyValues = ImmutableMap
        .of("device", "Example.Device", "property", "ExampleNewProperty");
Entity newEntityData = entityData.toBuilder().entityKeyValues(newKeyValues).build();

// Perform the update
Set<Entity> updatedEntities = entityService.updateEntities(Sets.newHashSet(newEntityData));

Variable Service

Service responsible for retrieving data related to variables.

Note

For registration or update of variables a role with VARIABLE:WRITE permission is required. Please follow procedure described here to obtain required authorization.

Additionally available methods (on top of findAll, findOne and findAll()):

Variable create(Variable variableData)
Variable update(Variable variableData)

More info can be found in Javadoc for VariableService interface.

Examples

Find variables using pattern on name:

Java

VariableService variableService = ServiceClientFactory.createVariableService();

Set<Variable> variablesDataSet = variableService.findAll(
        Variables.suchThat().systemName().eq("CMW").and().variableName().like("SPS%"));

Find variables using pattern on description:

Java

VariableService variableService = ServiceClientFactory.createVariableService();

Set<Variable> variablesDataSet = variableService
        .findAll(Variables.suchThat().systemName().eq("CMW").and().description().like("%Current Beam Energy%"));

Create a new variable pointing to a given entity and field for an open time window. In case of existing variable update its description. Demonstrates usage of create and update methods:

Java

final String SYSTEM = "CMW";
final String DEMO_VARIABLE = "DEMO:VARIABLE";

VariableService variableService = ServiceClientFactory.createVariableService();

Optional<Variable> variable = variableService.findOne(Variables.suchThat()
        .systemName().eq(SYSTEM).and().variableName().eq(DEMO_VARIABLE));

if (variable.isPresent()) {
    Variable updatedVariable = variable.get().toBuilder().description("updated description").build();
    variableService.update(updatedVariable);
}
else {
    EntityService entityService = ServiceClientFactory.createEntityService();

    Entity entity = entityService.findOne(
            Entities.suchThat().systemName().eq(SYSTEM).and().keyValues().like("%myUniqueDeviceProperty%"))
            .orElseThrow(() -> new IllegalArgumentException("No such entity"));

    VariableConfig config = VariableConfig.builder().entityId(entity.getId()).fieldName("myFieldName")
            .validity(TimeWindow.between(null, null)).build();

    Variable variableData = Variable.builder().variableName(DEMO_VARIABLE)
            .configs(ImmutableSortedSet.of(config))
            .systemSpec(entity.getSystemSpec())
            .build();

    variableService.create(variableData);
}

Hierarchy Service

Service responsible for retrieving data related to hierarchies.

Note

For registration or update of hierarchies a role with HIERARCHY:WRITE permission is required. Please follow procedure described here to obtain required authorization.

Additional available methods (on top of findAll, findOne and findAll() ):

Hierarchy create(Hierarchy hierarchy)
Hierarchy update(Hierarchy hierarchy)
void deleteLeaf(long hierarchyId)

Set<Hierarchy> getTopLevel(SystemSpec system)
List<Hierarchy> getChainTo(Hierarchy hierarchy)

void setVariables(long hierarchyId, Set<Long> variableIds)
void addVariables(long hierarchyId, Set<Long> variableIds)
void removeVariables(long hierarchyId, Set<Long> variableIds)
Set<Variable> getVariables(long hierarchyId)

void setEntities(long hierarchyId, Set<Long> entityIds)
void addEntities(long hierarchyId, Set<Long> entityIds)
void removeEntities(long hierarchyId, Set<Long> entityIds)
Set<Entity> getEntities(long hierarchyId)

More info can be found in Javadoc for HierarchyService interface.

Examples

Find hierarchies using pattern on path:

Java

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();

Set<Hierarchy> hierarchies = hierarchyService.findAll(Hierarchies.suchThat().path().like("%example%"));

Attach a "new node" to "/example" hierarchy. Once created, modify its description:

Java

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();
String systemName = "CMW";

SystemSpec systemSpec = ServiceClientFactory.createSystemSpecService().findByName(systemName)
        .orElseThrow(() -> new IllegalArgumentException("No such system specification"));

Hierarchy parent = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and().path().eq("/example"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy"));


Hierarchy newHierarchy = Hierarchy.builder().name("new node").description("New node description")
        .systemSpec(systemSpec).parent(parent).build();

Hierarchy modified = hierarchyService.create(newHierarchy);

modified.toBuilder().description("Modified node description").build();

hierarchyService.update(modified);

Delete "new node":

Java

hierarchyService.deleteLeaf(hierarchyService.findOne(Hierarchies.suchThat()
        .systemName().eq(systemName).and().path().eq("/example/new node"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy")));

Attach a variable to a hierarchy node. Demonstrates usage of findOne, addVariables and getVariables methods:

Java

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();
VariableService variableService = ServiceClientFactory.createVariableService();
String systemName = "CMW";

Hierarchy hierarchy = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and().path().eq("/example"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy"));

// Prepare variable to be attached to the hierarchy node
Variable variable = variableService
        .findOne(Variables.suchThat().systemName().eq(systemName).and().variableName().eq("DEMO:VARIABLE"))
        .orElseThrow(() -> new IllegalArgumentException("No such variable"));

// The variable will be added to the set of already attached variables
hierarchyService.addVariables(hierarchy.getId(), Collections.singleton(variable.getId()));

Set<Variable> variables = hierarchyService.getVariables(hierarchy.getId());

Replace entities attached to a hierarchy node with a single entity. Demonstrates usage of findAll, setEntities and getEntities methods:

Java

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();
EntityService entityService = ServiceClientFactory.createEntityService();
String systemName = "CMW";

Hierarchy hierarchy = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and().path().eq("/example"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy"));

// Obtain an entity of your interest
Entity entity = Iterables.getLast(entityService
        .findAll(Entities.suchThat().systemName().eq(systemName)));

// The entity will replace a set of already attached entities
hierarchyService.setEntities(hierarchy.getId(), Collections.singleton(entity.getId()));

Set<Entity> entities = hierarchyService.getEntities(hierarchy.getId());

Please not that the methods setVariables and setEntities work as normal setters - they replace the current state. Be sure not to overwrite anything you don't intend to.

Cache Configurations

NXCALS is using Caffeine as its library to handle in-memory caching. Below, we can find a list of all the user-configurable properties of the cache with their default value.

cache.spec.initial.capacity = 500
cache.spec.maximum.size = Integer.MAX_VALUE 
cache.spec.expire.after.access.seconds = Integer.MAX_VALUE
cache.spec.expire.after.write.seconds = Integer.MAX_VALUE

Internally, we are using Typesafe and the following properties are set in a typesafe config way. In case you need it, you can configure any properties in two ways. Either you overwrite those properties in your application.conf file, for example by adding:

cache.spec {
  initial.capacity = 250,
  maximum.size = 1000,
  expire.after.access.seconds = 600,
  expire.after.write.seconds = 600
}

or by specifying those properties as JVM properties, for example:

-Dcache.spec.initial.capacity=250 -Dcache.spec.maximum.size=1000

For the temporal settings, by default we are considering seconds, so the unit of measure is not needed after the specified value.

Important

There are some constraints that need to be taken into account for the cache not to slow down too much NXCALS. Minimum cache.spec.initial.capacity = 100
Minimum cache.spec.maximum.size = 500
Minimum cache.spec.expire.after.access.seconds = 600
Minimum cache.spec.expire.after.write.seconds = 600

Broking one of these constraints will make NXCALS throw an exception. If you are not sure what to set for a certain property, you can omit it to use the default one.