Skip to content

Metadata API

NXCALS meta-data can be retrieved using Java Client API from cern.nxcals.api.metadata package.

NXCALS Service Client API itself consist of four end points:

Full Java documentation is available here.

In contrary to versions prior to the 0.2.0 the API of each endpoint contains similar search methods that accept arbitrary Condition built with a query builder allowing to search for one or many objects in a very flexible way. The method finding one object (findOne and findById) are cached. 

Set<X> findAll(Condition<Xs>)
Optional<X> findOne(Condition<Xs>)
Optional<X> findById(long id)

Please note that you will find corresponding Condition builders with classes that names are made plural from the domain object. So Entity -> Entities, Variable -> Variables, etc.

In order to build a Condition please use the following code: 

Entities.suchThat().systemId().eq(1L).and().keyValues().like("%myDevice%");

Note that conditions are by default in case-sensitive mode. Meaning:

Groups.suchThat().name().eq("myName");

Will only match group named "myName".

If you wish to enable case-insensitive search please use caseInsensitive() call prior to any string operation:

Groups.suchThat().name().caseInsensitive().eq("myName");

This will match every possible combination of casing of "myName", eg. "MYNAME", "MyName", "MyNaMe" etc.

Case insensitive search is also available for key-value searches: 

Entities.suchThat().keyValues().caseInsensitive().like("%myDevice%myProperty%");

Conditions must return a correct number of records according to the method used. If a Condition that returns multiple rows is passed to the findOne method an exception will be thrown (IllegalArgumentException).

Important

Using case insensitive search might cause a query to return more than one value in unexpected circumstances. Be sure to call queryAll method for safety, whenever opting for case-insensitive search.

Warning

Running Python code for this API requires a specific setup as described here. The code snippets provided on this page are using JPype.

System Service

Service responsible for retrieving data related to systems.

Additionally available methods (on top of findAll, findOne and findById()):

Optional<System> findByName(String name)
Set<System> findAll()

More info can be found in Javadoc for SystemSpecService interface.

Examples

Find information about system using its name

SystemSpecService systemService = ServiceClientFactory.createSystemSpecService();

SystemSpec systemData = systemService.findByName("CMW")
        .orElseThrow(() -> new IllegalArgumentException("No such system"));

systemService = spark._jvm.cern.nxcals.api.extraction.metadata.ServiceClientFactory.createSystemSpecService()

systemData = systemService.findByName("CMW")

if systemData.isEmpty():
    raise ValueError("No such system")

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
systemService = ServiceClientFactory.createSystemSpecService()

systemData = systemService.findByName("CMW")

if systemData.isEmpty():
    raise ValueError("No such system")

Entity Service

Service responsible for retrieving data related to Entities.

Note

For update of entities a role with WRITE permission for a given system is required. Please follow procedure described here to obtain required authorization.

In addition to findAll, findOne and findById(), the EntityService exposes two additional methods for obtaining the Entity history(ies) for a given time window: 

Set<Entity> findAllWithHistory(Condition<Entities> condition, long historyStartTime, long historyEndTime)
Optional<Entity> findOneWithHistory(Condition<Entities> condition, long historyStartTime, long historyEndTime)

There are some other methods for data persistence / manipulation: 

Entity createEntity(long systemId, Map<String, Object> entityKey, Map<String, Object> partitionKey)
Set<Entity> updateEntities(Set<Entity> entityDataList)

More info can be found in Javadoc for EntityService interface.

withOptions

withOptions allows to specify additional options to query. Options common with Variables are:

  • limit(int size) - allows to limit number of returned Entity objects;
  • orderBy - allows to sort objects by specified field in Entity. Order might be ascending (asc) or descending (desc);
  • offset(int offset) - enables skipping first N result, what can be used in combination with above.

Entities contains following specific methods:

  • noHistory - allows to query just for plain Entity objects, without EntityHistory;
  • withHistory(long/String/Instant start, long/String/Instant end) - allows to query for Entity objects with history entries, which were valid in specified time range.

By default, query will return Entity objects with the latest EntityHistory.

Examples

Search for one entity.

Please note that extracted entity history contains only a current history:

EntityService entityService = ServiceClientFactory.createEntityService();
SystemSpecService systemService = ServiceClientFactory.createSystemSpecService();

SystemSpec systemData = systemService.findOne(SystemSpecs.suchThat().name().eq("CMW"))
        .orElseThrow(() -> new IllegalArgumentException("System not found"));

Map<String, Object> keyValues = ImmutableMap.of("device", "LHC.LUMISERVER", "property", "CrossingAngleIP1");
Entity entityData = entityService.findOne(
        Entities.suchThat().systemId().eq(systemData.getId()).and().keyValues().eq(systemData, keyValues))
        .orElseThrow(() -> new IllegalArgumentException("Entity not found"));

_nxcals_api = spark._jvm.cern.nxcals.api
entityService = _nxcals_api.extraction.metadata.ServiceClientFactory.createEntityService()
systemService = _nxcals_api.extraction.metadata.ServiceClientFactory.createSystemSpecService()
SystemSpecs = _nxcals_api.extraction.metadata.queries.SystemSpecs
Entities = _nxcals_api.extraction.metadata.queries.Entities

systemData = systemService.findOne(SystemSpecs.suchThat().name().eq("CMW"))


if systemData.isEmpty():
    raise ValueError("System not found")

keyValues = {"device": "LHC.LUMISERVER", "property": "CrossingAngleIP1"}

entityData = entityService.findOne(getattr(Entities.suchThat().systemId().eq(systemData.get().getId()),'and')().keyValues().eq(systemData.get(), keyValues))

if entityData.isEmpty():
    raise ValueError("Entity not found")

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import SystemSpecs, Entities
from com.google.common.collect import ImmutableMap

entityService = ServiceClientFactory.createEntityService()
systemService = ServiceClientFactory.createSystemSpecService()

systemData = systemService.findOne(SystemSpecs.suchThat().name().eq("CMW"))

if systemData.isEmpty():
    raise ValueError("System not found")

keyValues = ImmutableMap.of("device", "LHC.LUMISERVER", "property", "CrossingAngleIP1")

entityData = entityService.findOne( Entities.suchThat().systemId().eq(systemData.get().getId()).and_().keyValues().eq(systemData.get(), keyValues))

if entityData.isEmpty():
    raise ValueError("Entity not found")

Search for an entity with its history which is present in the provided time range.

For the example below, two entity histories will be shown: first with the validity from 1970-01-01T00:00:00Z until 2019-01-01T00:00:00Z, and second with the validity from 2019-01-01T00:00:00Z onwards:

SystemSpec systemData = ServiceClientFactory.createSystemSpecService()
        .findOne(SystemSpecs.suchThat().name().eq("CMW"))
        .orElseThrow(() -> new IllegalArgumentException("System not found"));

Map<String, Object> keyValues = ImmutableMap.of("device", "LHC.LUMISERVER", "property", "CrossingAngleIP1");

EntityService entityService = ServiceClientFactory.createEntityService();
Entity entities = entityService.findOneWithHistory(
        Entities.suchThat().systemName().eq(systemData.getName()).and().keyValues().eq(systemData, keyValues),
        TimeUtils.getNanosFromString("2017-10-10 14:15:00.000000000"),
        TimeUtils.getNanosFromString("2020-10-26 14:15:00.000000000")
).orElseThrow(() -> new IllegalArgumentException("Entity not found"));

entities.getEntityHistory().forEach(System.out::println);

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
SystemSpecs = _nxcals_api.extraction.metadata.queries.SystemSpecs
Entities = _nxcals_api.extraction.metadata.queries.Entities
TimeUtils = _nxcals_api.utils.TimeUtils

systemData = ServiceClientFactory.createSystemSpecService().findOne(SystemSpecs.suchThat().name().eq("CMW"))

if systemData.isEmpty():
    raise ValueError("System not found")

keyValues = {"device": "LHC.LUMISERVER", "property": "CrossingAngleIP1"}

entityService = ServiceClientFactory.createEntityService()

entities = entityService.findOneWithHistory(getattr(
    Entities.suchThat().systemName().eq(systemData.get().getName()), 'and')().keyValues().eq(systemData.get(), keyValues),
                                            TimeUtils.getNanosFromString("2017-10-10 14:15:00.000000000"),
                                            TimeUtils.getNanosFromString("2020-10-26 14:15:00.000000000"))

if entities.isEmpty():
    raise ValueError("Entity not found")

for entityHistory in entities.get().getEntityHistory():
    print(entityHistory)

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import SystemSpecs, Entities
from com.google.common.collect import ImmutableMap
from cern.nxcals.api.utils import TimeUtils

systemData = ServiceClientFactory.createSystemSpecService().findOne(SystemSpecs.suchThat().name().eq("CMW"))

if systemData.isEmpty():
    raise ValueError("System not found")

keyValues = ImmutableMap.of("device", "LHC.LUMISERVER", "property", "CrossingAngleIP1")

entityService = ServiceClientFactory.createEntityService()

entities = entityService.findOneWithHistory(
    Entities.suchThat().systemName().eq(systemData.get().getName()).and_().keyValues().eq(systemData.get(), keyValues),
    TimeUtils.getNanosFromString("2017-10-10 14:15:00.000000000"),
    TimeUtils.getNanosFromString("2020-10-26 14:15:00.000000000"))

if entities.isEmpty():
    raise ValueError("Entity not found")

for entityHistory in entities.get().getEntityHistory():
    print(entityHistory)

Update entity (change its key values)

SystemSpecService systemService = ServiceClientFactory.createSystemSpecService();
EntityService entityService = ServiceClientFactory.createEntityService();

SystemSpec systemData = systemService.findByName("CMW")
        .orElseThrow(() -> new IllegalArgumentException("No such system CMW"));

Map<String, Object> keyValues = ImmutableMap.of("device", "MY.DEVICE", "property", "MyProperty");
Entity entityData = entityService.findOne(Entities.suchThat()
        .systemId().eq(systemData.getId())
        .and()
        .keyValues().eq(systemData, keyValues))
        .orElseThrow(() -> new IllegalArgumentException("Entity not found"));

// Change entity to set new keyValues
Map<String, Object> newKeyValues = ImmutableMap
        .of("device", "Example.Device", "property", "ExampleNewProperty");
Entity newEntityData = entityData.toBuilder().entityKeyValues(newKeyValues).build();

// Perform the update
Set<Entity> updatedEntities = entityService.updateEntities(Sets.newHashSet(newEntityData));

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Entities = _nxcals_api.extraction.metadata.queries.Entities

entityService = ServiceClientFactory.createEntityService()
systemService = ServiceClientFactory.createSystemSpecService()

systemData = systemService.findByName("CMW")

if systemData.isEmpty():
    raise ValueError("No such system CMW")

keyValues = {"device": "MY.DEVICE", "property": "MyProperty"}

entityData = entityService.findOne(getattr(Entities.suchThat().systemId().eq(systemData.get().getId()), 'and')().keyValues().eq(systemData.get(), keyValues))

if entityData.isEmpty():
    raise ValueError("Entity not found")

# Change entity to set new keyValues
newKeyValues = {"device": "Example.Device", "property": "ExampleNewProperty"}
newEntityData = entityData.get().toBuilder().entityKeyValues(newKeyValues).build()

# Perform the update
updatedEntities = entityService.updateEntities({newEntityData})

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Entities
from com.google.common.collect import ImmutableMap
from com.google.common.collect import Sets

entityService = ServiceClientFactory.createEntityService()
systemService = ServiceClientFactory.createSystemSpecService()

systemData = systemService.findByName("CMW")

if systemData.isEmpty():
    raise ValueError("No such system CMW")

keyValues = ImmutableMap.of("device", "MY.DEVICE", "property", "MyProperty")

entityData = entityService.findOne( Entities.suchThat().systemId().eq(systemData.get().getId()).and_().keyValues().eq(systemData.get(), keyValues))

if entityData.isEmpty():
    raise ValueError("Entity not found")

# Change entity to set new keyValues
newKeyValues = ImmutableMap.of("device", "Example.Device", "property", "ExampleNewProperty")
newEntityData = entityData.get().toBuilder().entityKeyValues(newKeyValues).build()

# Perform the update
updatedEntities = entityService.updateEntities(Sets.newHashSet(newEntityData))

Search for multiple entities withOptions: limit, orderBy and withHistory

SystemSpec systemData = ServiceClientFactory.createSystemSpecService()
        .findByName("CMW")
        .orElseThrow(() -> new IllegalArgumentException("System not found"));
Map<String, Object> keyValues = ImmutableMap.of("device", "LHC.LUMISERVER", "property", "CrossingAngleIP%");

EntityService entityService = ServiceClientFactory.createEntityService();

EntityQueryWithOptions query = Entities.suchThat()
        .keyValues().like(systemData, keyValues)
        .withOptions()
        .withHistory("2017-10-10 14:15:00.000000000", "2020-10-26 14:15:00.000000000")
        .limit(2)
        .orderBy().keyValues().desc();

List<Entity> entities = entityService.findAll(query);

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Entities = _nxcals_api.metadata.queries.Entities  # New package!

entity_service = ServiceClientFactory.createEntityService()
system_service = ServiceClientFactory.createSystemSpecService()

system_data = system_service.findByName("CMW")

if system_data.isEmpty():
    raise ValueError("No such system CMW")

key_values = {"device": "LHC.LUMISERVER", "property": "CrossingAngleIP%"}
query = Entities.suchThat() \
    .keyValues().like(system_data.get(), key_values) \
    .withOptions() \
    .withHistory("2017-10-10 14:15:00.000000000", "2020-10-26 14:15:00.000000000") \
    .limit(2) \
    .orderBy().keyValues().desc()

entities = entity_service.findAll(query)

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.metadata.queries import Entities  # new package!
from com.google.common.collect import ImmutableMap

entityService = ServiceClientFactory.createEntityService()
systemService = ServiceClientFactory.createSystemSpecService()

systemData = systemService.findByName("CMW")

if systemData.isEmpty():
    raise ValueError("No such system CMW")

keyValues = ImmutableMap.of("device", "LHC.LUMISERVER", "property", "CrossingAngleIP%")

query = Entities.suchThat() \
    .keyValues().like(systemData.get(), keyValues) \
    .withOptions() \
    .withHistory("2017-10-10 14:15:00.000000000", "2020-10-26 14:15:00.000000000") \
    .limit(2) \
    .orderBy().keyValues().desc()

entityData = entityService.findAll(query)

Variable Service

Service responsible for retrieving data related to variables.

Note

For registration or update of variables a role with VARIABLE:WRITE permission is required. Please follow procedure described here to obtain required authorization.

Additionally available methods (on top of findAll, findOne and findById()):

Variable create(Variable variableData)
Variable update(Variable variableData)
List<Variable> findAll(VariableQueryWithOptions query)
Optional<Variable> findOne(VariableQueryWithOptions query)

More info can be found in Javadoc for VariableService interface.

withOptions

withOptions allows to specify additional options to query. Options common with Entities are:

  • limit(int size) - allows to limit number of returned Variable objects;
  • orderBy() - allows to sort objects by specified field in Variable. Order might be ascending (asc) or descending (desc);
  • offset(int offset) - enables skipping first N result, what can be used in combination with above.

Currently Variables contains one specific method - noConfigs which allows to query just for plain Variable objects, without VariableConfig. By default, query will return Variable objects with VariableConfig.

Examples

Find variables using pattern on name

VariableService variableService = ServiceClientFactory.createVariableService();

Set<Variable> variablesDataSet = variableService.findAll(
        Variables.suchThat().systemName().eq("CMW").and().variableName().like("SPS%"));

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Variables = _nxcals_api.extraction.metadata.queries.Variables

variableService = ServiceClientFactory.createVariableService()

variables = variableService.findAll(
    getattr(Variables.suchThat().systemName().eq("CMW"), 'and')().variableName().like("SPS%"))

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Variables

variableService = ServiceClientFactory.createVariableService()

variables = variableService.findAll(
    Variables.suchThat().systemName().eq("CMW").and_().variableName().like("SPS%"))

Find variables using pattern on description

VariableService variableService = ServiceClientFactory.createVariableService();

Set<Variable> variablesDataSet = variableService
        .findAll(Variables.suchThat().systemName().eq("CMW").and().description().like("%Current Beam Energy%"));

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Variables = _nxcals_api.extraction.metadata.queries.Variables

variableService = ServiceClientFactory.createVariableService()

variables = variableService.findAll(
    getattr(Variables.suchThat().systemName().eq("CMW"), 'and')().description().like("%Current Beam Energy%"))

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Variables

variableService = ServiceClientFactory.createVariableService()

variables = variableService.findAll(
    Variables.suchThat().systemName().eq("CMW").and_().description().like("%Current Beam Energy%"))

Create a new variable pointing to a given entity and field for an open time window. In case of existing variable update its description. Demonstrates usage of create and update methods:

final String SYSTEM = "CMW";
final String DEMO_VARIABLE = "DEMO:VARIABLE";

VariableService variableService = ServiceClientFactory.createVariableService();

Optional<Variable> variable = variableService.findOne(Variables.suchThat()
        .systemName().eq(SYSTEM).and().variableName().eq(DEMO_VARIABLE));

if (variable.isPresent()) {
    Variable updatedVariable = variable.get().toBuilder().description("updated description").build();
    variableService.update(updatedVariable);
}
else {
    EntityService entityService = ServiceClientFactory.createEntityService();

    Entity entity = entityService.findOne(
            Entities.suchThat().systemName().eq(SYSTEM).and().keyValues().like("%myUniqueDeviceProperty%"))
            .orElseThrow(() -> new IllegalArgumentException("No such entity"));

    VariableConfig config = VariableConfig.builder().entityId(entity.getId()).fieldName("myFieldName")
            .validity(TimeWindow.between(null, null)).build();

    Variable variableData = Variable.builder().variableName(DEMO_VARIABLE)
            .configs(ImmutableSortedSet.of(config))
            .systemSpec(entity.getSystemSpec())
            .build();

    variableService.create(variableData);
}

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Variables = _nxcals_api.extraction.metadata.queries.Variables
Entities = _nxcals_api.extraction.metadata.queries.Entities
VariableConfig = _nxcals_api.domain.VariableConfig
TimeWindow = _nxcals_api.domain.TimeWindow
Variable = _nxcals_api.domain.Variable
ImmutableSortedSet = spark._jvm.com.google.common.collect.ImmutableSortedSet

SYSTEM = "CMW"
DEMO_VARIABLE = "DEMO:VARIABLE"

variableService = ServiceClientFactory.createVariableService()

variable = variableService.findOne(getattr(Variables.suchThat().systemName().eq(SYSTEM), 'and')().variableName().eq(DEMO_VARIABLE))

if variable.isPresent():

    updatedVariable = variable.get().toBuilder().description("updated description").build()
    variableService.update(updatedVariable)

else:
    entityService = ServiceClientFactory.createEntityService()

    entity = entityService.findOne(getattr(Entities.suchThat().systemName().eq(SYSTEM), 'and')().keyValues().like("%myUniqueDeviceProperty%"))

    if entity.isEmpty():
        raise ValueError("No such entity")

    config = VariableConfig.builder().entityId(entity.get().getId()).fieldName("myFieldName").validity(
        TimeWindow.between(None, None)).build()

    variableData = Variable.builder().variableName(DEMO_VARIABLE).configs(
        ImmutableSortedSet.of(config)).systemSpec(entity.get().getSystemSpec()).build()

    variableService.create(variableData)

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Variables, Entities
from cern.nxcals.api.domain import VariableConfig, TimeWindow, Variable
from com.google.common.collect import ImmutableSortedSet

SYSTEM = "CMW"
DEMO_VARIABLE = "DEMO:VARIABLE"

variableService = ServiceClientFactory.createVariableService()

variable = variableService.findOne(Variables.suchThat().systemName().eq(SYSTEM).and_().variableName().eq(DEMO_VARIABLE))

if variable.isPresent():

    updatedVariable = variable.get().toBuilder().description("updated description").build()
    variableService.update(updatedVariable)

else:
    entityService = ServiceClientFactory.createEntityService()

    entity = entityService.findOne(Entities.suchThat().systemName().eq(SYSTEM).and_().keyValues().like("%myUniqueDeviceProperty%"))

    if entity.isEmpty():
        raise ValueError("No such entity")

    config = VariableConfig.builder().entityId(entity.get().getId()).fieldName("myFieldName").validity(
        TimeWindow.between(None, None)).build()

    variableData = Variable.builder().variableName(DEMO_VARIABLE).configs(
        ImmutableSortedSet.of(config)).systemSpec(entity.get().getSystemSpec()).build()

    variableService.create(variableData)

Get list of variables sorted by name without configs

VariableService variableService = ServiceClientFactory.createVariableService();

VariableQueryWithOptions query = Variables.suchThat()
        .variableName().like("SPS%")
        .withOptions()
        .noConfigs()
        .orderBy().variableName().asc()
        .limit(10);

List<Variable> variables = variableService.findAll(query);

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Variables = _nxcals_api.metadata.queries.Variables  # New package!

variable_service = ServiceClientFactory.createVariableService()

variables = variable_service.findAll(Variables.suchThat()
                                     .variableName().like("SPS%")
                                     .withOptions()
                                     .noConfigs()
                                     .orderBy().variableName().asc()
                                     .limit(10))

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.metadata.queries import Variables  # New package!

variableService = ServiceClientFactory.createVariableService()

variables = variableService.findAll(Variables.suchThat()
                                    .variableName().like("SPS%")
                                    .withOptions()
                                    .noConfigs()
                                    .orderBy().variableName().asc()
                                    .limit(10))

Hierarchy Service

Service responsible for retrieving data related to hierarchies.

Note

For registration or update of hierarchies a role with HIERARCHY:WRITE permission is required. Please follow procedure described here to obtain required authorization.

Additional available methods (on top of findAll, findOne and findAll() ):

Hierarchy create(Hierarchy hierarchy)
Hierarchy update(Hierarchy hierarchy)
void deleteLeaf(long hierarchyId)

Set<Hierarchy> getTopLevel(SystemSpec system)
List<Hierarchy> getChainTo(Hierarchy hierarchy)

void setVariables(long hierarchyId, Set<Long> variableIds)
void addVariables(long hierarchyId, Set<Long> variableIds)
void removeVariables(long hierarchyId, Set<Long> variableIds)
Set<Variable> getVariables(long hierarchyId)

void setEntities(long hierarchyId, Set<Long> entityIds)
void addEntities(long hierarchyId, Set<Long> entityIds)
void removeEntities(long hierarchyId, Set<Long> entityIds)
Set<Entity> getEntities(long hierarchyId)

More info can be found in Javadoc for HierarchyService interface.

Examples

Find hierarchies using pattern on path

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();

Set<Hierarchy> hierarchies = hierarchyService.findAll(Hierarchies.suchThat().path().like("%example%"));

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Hierarchies = _nxcals_api.extraction.metadata.queries.Hierarchies

hierarchyService = ServiceClientFactory.createHierarchyService()

hierarchies = hierarchyService.findAll(Hierarchies.suchThat().path().like("%example%"))

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Hierarchies

hierarchyService = ServiceClientFactory.createHierarchyService()

hierarchies = hierarchyService.findAll(Hierarchies.suchThat().path().like("%example%"))

Attach a "new node" to "/example" hierarchy. Once created, modify its description

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();
String systemName = "CMW";

SystemSpec systemSpec = ServiceClientFactory.createSystemSpecService().findByName(systemName)
        .orElseThrow(() -> new IllegalArgumentException("No such system specification"));

Hierarchy parent = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and().path().eq("/example"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy"));


Hierarchy newHierarchy = Hierarchy.builder().name("new node").description("New node description")
        .systemSpec(systemSpec).parent(parent).build();

Hierarchy modified = hierarchyService.create(newHierarchy);

modified.toBuilder().description("Modified node description").build();

hierarchyService.update(modified);

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Hierarchies = _nxcals_api.extraction.metadata.queries.Hierarchies
Hierarchy = _nxcals_api.domain.Hierarchy

hierarchyService = ServiceClientFactory.createHierarchyService()
systemName = "CMW"

systemSpec = ServiceClientFactory.createSystemSpecService().findByName(systemName)

if systemSpec.isEmpty():
    raise ValueError("No such system specification")

parent = hierarchyService.findOne(getattr(Hierarchies.suchThat().systemName().eq(systemName), 'and')().path().eq("/example"))

if parent.isEmpty():
    raise ValueError("No such hierarchy")

newHierarchy = Hierarchy.builder().name("new node").description("New node description").systemSpec(systemSpec.get()).parent(parent.get()).build()

modified = hierarchyService.create(newHierarchy)

modified.toBuilder().description("Modified node description").build()

hierarchyService.update(modified)

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Hierarchies
from cern.nxcals.api.domain import Hierarchy

hierarchyService = ServiceClientFactory.createHierarchyService()
systemName = "CMW"

systemSpec = ServiceClientFactory.createSystemSpecService().findByName(systemName)

if systemSpec.isEmpty():
    raise ValueError("No such system specification")

parent = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and_().path().eq("/example"))

if parent.isEmpty():
    raise ValueError("No such hierarchy")

newHierarchy = Hierarchy.builder().name("new node").description("New node description").systemSpec(systemSpec.get()).parent(parent.get()).build()

modified = hierarchyService.create(newHierarchy)

modified.toBuilder().description("Modified node description").build()

hierarchyService.update(modified)

Delete new node

hierarchyService.deleteLeaf(hierarchyService.findOne(Hierarchies.suchThat()
        .systemName().eq(systemName).and().path().eq("/example/new node"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy")));

leaf = hierarchyService.findOne(getattr(Hierarchies.suchThat().systemName().eq(systemName), 'and')().path().eq("/example/new node"))
if leaf.isEmpty():
    raise ValueError("No such hierarchy")

hierarchyService.deleteLeaf(leaf.get())

leaf = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and_().path().eq("/example/new node"))
if leaf.isEmpty():
    raise ValueError("No such hierarchy")

hierarchyService.deleteLeaf(leaf.get())

Attach a variable to a hierarchy node. Demonstrates usage of findOne, addVariables and getVariables methods

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();
VariableService variableService = ServiceClientFactory.createVariableService();
String systemName = "CMW";

Hierarchy hierarchy = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and().path().eq("/example"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy"));

// Prepare variable to be attached to the hierarchy node
Variable variable = variableService
        .findOne(Variables.suchThat().systemName().eq(systemName).and().variableName().eq("DEMO:VARIABLE"))
        .orElseThrow(() -> new IllegalArgumentException("No such variable"));

// The variable will be added to the set of already attached variables
hierarchyService.addVariables(hierarchy.getId(), Collections.singleton(variable.getId()));

Set<Variable> variables = hierarchyService.getVariables(hierarchy.getId());

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Hierarchies = _nxcals_api.extraction.metadata.queries.Hierarchies
variables = _nxcals_api.extraction.metadata.queries.Variables
Collections = spark._jvm.java.util.Collections

hierarchyService = ServiceClientFactory.createHierarchyService()
variableService = ServiceClientFactory.createVariableService()

systemName = "CMW"

hierarchy = hierarchyService.findOne(getattr(Hierarchies.suchThat().systemName().eq(systemName), 'and')().path().eq("/example"))

if hierarchy.isEmpty():
    raise ValueError("No such hierarchy")

# Prepare variable to be attached to the hierarchy node
variable = variableService.findOne(getattr(Variables.suchThat().systemName().eq(systemName), 'and')().variableName().eq("DEMO:VARIABLE"))

if variable.isEmpty():
    raise ValueError("No such variable")

# The variable will be added to the set of already attached variables
hierarchyService.addVariables(hierarchy.get().getId(), Collections.singleton(variable.get().getId()))

variables = hierarchyService.getVariables(hierarchy.get().getId())

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Hierarchies, Variables
from java.util import Collections

hierarchyService = ServiceClientFactory.createHierarchyService()
variableService = ServiceClientFactory.createVariableService()

systemName = "CMW"

hierarchy = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(systemName).and_().path().eq("/example"))

if hierarchy.isEmpty():
    raise ValueError("No such hierarchy")

# Prepare variable to be attached to the hierarchy node
variable = variableService.findOne(Variables.suchThat().systemName().eq(systemName).and_().variableName().eq("DEMO:VARIABLE"))

if variable.isEmpty():
    raise ValueError("No such variable")

# The variable will be added to the set of already attached variables
hierarchyService.addVariables(hierarchy.get().getId(), Collections.singleton(variable.get().getId()))

variables = hierarchyService.getVariables(hierarchy.get().getId())

Replace entities attached to a hierarchy node with a single entity. Demonstrates usage of findAll, setEntities and getEntities methods

HierarchyService hierarchyService = ServiceClientFactory.createHierarchyService();
EntityService entityService = ServiceClientFactory.createEntityService();
String hierarchySystemName = "CERN";
String systemName = "CMW";

Hierarchy hierarchy = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(hierarchySystemName).and().path().eq("/example"))
        .orElseThrow(() -> new IllegalArgumentException("No such hierarchy"));

// Obtain an entity of your interest
Entity entity = Iterables.getLast(entityService
        .findAll(Entities.suchThat().systemName().eq(systemName)));

// The entity will replace a set of already attached entities
hierarchyService.setEntities(hierarchy.getId(), Collections.singleton(entity.getId()));

Set<Entity> entities = hierarchyService.getEntities(hierarchy.getId());

_nxcals_api = spark._jvm.cern.nxcals.api
ServiceClientFactory = _nxcals_api.extraction.metadata.ServiceClientFactory
Entities = _nxcals_api.extraction.metadata.queries.Entities
Hierarchies = _nxcals_api.extraction.metadata.queries.Hierarchies
Collections = spark._jvm.java.util.Collections
Iterables = spark._jvm.com.google.common.collect.Iterables

hierarchyService = ServiceClientFactory.createHierarchyService()
entityService = ServiceClientFactory.createEntityService()
hierarchySystemName = "CERN"
systemName = "CMW"

hierarchy = hierarchyService.findOne(
    getattr(Hierarchies.suchThat().systemName().eq(hierarchySystemName), 'and')().path().eq("/example"))

if hierarchy.isEmpty():
    raise ValueError("No such hierarchy")

# Obtain an entity of your interest
entity = Iterables.getLast(entityService
                           .findAll(Entities.suchThat().systemName().eq(systemName)))
print(entity)

# The entity will replace a set of already attached entities
hierarchyService.setEntities(hierarchy.get().getId(), Collections.singleton(entity.getId()))

entities = hierarchyService.getEntities(hierarchy.get().getId())
print(entities)

from cern.nxcals.api.extraction.metadata import ServiceClientFactory
from cern.nxcals.api.extraction.metadata.queries import Entities, Hierarchies
from java.util import Collections
from com.google.common.collect import Iterables

hierarchyService = ServiceClientFactory.createHierarchyService()
entityService = ServiceClientFactory.createEntityService()
hierarchySystemName = "CERN"
systemName = "CMW"

hierarchy = hierarchyService.findOne(Hierarchies.suchThat().systemName().eq(hierarchySystemName).and_().path().eq("/example"))

if hierarchy.isEmpty():
    raise ValueError("No such hierarchy")

# Obtain an entity of your interest
entity = Iterables.getLast(entityService
                          .findAll(Entities.suchThat().systemName().eq(systemName)))
print(entity)

# The entity will replace a set of already attached entities
hierarchyService.setEntities(hierarchy.get().getId(), Collections.singleton(entity.getId()))

entities = hierarchyService.getEntities(hierarchy.get().getId())
print(entities)

Please not that the methods setVariables and setEntities work as normal setters - they replace the current state. Be sure not to overwrite anything you don't intend to.

Cache Configurations

NXCALS is using Caffeine as its library to handle in-memory caching. Below, we can find a list of all the user-configurable properties of the cache with their default value. 

cache.spec.initial.capacity = 500
cache.spec.maximum.size = Integer.MAX_VALUE 
cache.spec.expire.after.access.seconds = Integer.MAX_VALUE
cache.spec.expire.after.write.seconds = Integer.MAX_VALUE
Internally, we are using Typesafe and the following properties are set in a typesafe config way. In case you need it, you can configure any properties in two ways. Either you overwrite those properties in your application.conf file, for example by adding: 
cache.spec {
  initial.capacity = 250,
  maximum.size = 1000,
  expire.after.access.seconds = 600,
  expire.after.write.seconds = 600
}
or by specifying those properties as JVM properties, for example: 
-Dcache.spec.initial.capacity=250 -Dcache.spec.maximum.size=1000
For the temporal settings, by default we are considering seconds, so the unit of measure is not needed after the specified value.

Important

There are some constraints that need to be taken into account for the cache not to slow down too much NXCALS. Minimum cache.spec.initial.capacity = 100
Minimum cache.spec.maximum.size = 500
Minimum cache.spec.expire.after.access.seconds = 600
Minimum cache.spec.expire.after.write.seconds = 600

Broking one of these constraints will make NXCALS throw an exception. If you are not sure what to set for a certain property, you can omit it to use the default one.

Additional options (for Entities and Variables) - limit, offset, and orderBy

There is possibility to limit number of returned objects. You can do this using at first withOptions(), and then limit(). Example: 

Entities.suchThat().systemId().eq(1L).withOptions().limit(10);

Apart from limit, there is also way to specify offset(). It causes, that first N records are skipped. With combination with limit, it can be used to provide pagination. 

Entities.suchThat().systemId().eq(1L).withOptions().offset(10).limit(10);

Another option is sorting records by chosen field: 

Entities.suchThat().systemId().eq(1L).withOptions().orderBy().keyValues();