dmytrolugovyi's profile

Monday, February 22nd, 2021 1:38 PM

OMRE (Operating Model Reverse Engineering)

This topic is created to discuss the current version of the OMRE (https://marketplace.collibra.com/listings/operating-model-reverse-engineering-omre/) as well as the possible improvements that could make this solution better.

3 years ago

Hi Dmytro, all

Could you explain the tags ‘Used_By_OMRE’ and ‘Part_Of_OMRE’ since it is not part of the documentation? (as far as I can see).

Br,

Martijn

43 Messages

@martijn.datashift.eu OMRE solution has quite a few operating model objects which will be created during the installation process. At the same time, OMRE uses OOTB operating model objects where it’s possible. The tags have been added to provide the ability to control your OM excluding those which have been created by OMRE. ‘Used_By_OMRE’ is the OOTB objects which have been used by OMRE, ‘Part_Of_OMRE’ is the custom objects which have been added by OMRE.

Really clear thanks Dmytro!

19 Messages

3 years ago

Hello, today i tried to install the OMRE tool (Operating Model Reverse Engineering V1.2.0).
I am ‘on -premises’ with 5.7.7 DGC version.

I started to import file ‘OMRE_v1.1.0_Full.cma’ (in the DGC ‘Settings’ import menu) and the simulation report has failed.
In the DGC.log there is an error log (java stack trace) : workflowConfigurationVariableUnknown
Params: [usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType]
Can you please help ? This tool seems to be usefull and i would like to use it.
Regards, François
Complete stack trace is here :
2021-06-28 11:42:28.382 [pool-21-thread-1] ERROR c.c.d.m.a.i.c.impl.MigrationApiImpl - error during Migration []
com.collibra.common.api.exception.ApiIllegalArgumentException: com.collibra.common.exception.CollibraIllegalArgumentException: workflowConfigurationVariableUnknown
Params: [usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType]
at com.collibra.dgc.core.api.ApiExceptionTranslationHelper.translateToApiException(ApiExceptionTranslationHelper.java:57)
at com.collibra.dgc.core.api.aspect.ApiExceptionHandlingAspect.mapExceptions(ApiExceptionHandlingAspect.java:35)
at sun.reflect.GeneratedMethodAccessor838.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy482.changeWorkflowDefinition(Unknown Source)
at com.collibra.dgc.migration.api.internal.component.extensions.workflow.impl.XWorkflowDefinitionApiImpl.change(XWorkflowDefinitionApiImpl.java:135)
at com.collibra.dgc.migration.api.internal.component.extensions.workflow.impl.XWorkflowDefinitionApiImpl.add(XWorkflowDefinitionApiImpl.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at com.collibra.dgc.core.transaction.DGCApiV2Aspect.processDGCTransactionalAnnotation(DGCApiV2Aspect.java:45)
at sun.reflect.GeneratedMethodAccessor839.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy557.add(Unknown Source)
at com.collibra.dgc.migration.api.internal.component.impl.importers.workflow.WorkflowImporter.add(WorkflowImporter.java:170)
at com.collibra.dgc.migration.api.internal.component.impl.importers.workflow.WorkflowImporter.importWorkflowDefinition(WorkflowImporter.java:143)
at com.collibra.dgc.migration.api.internal.component.impl.importers.workflow.WorkflowImporter.performImport(WorkflowImporter.java:61)
at com.collibra.dgc.migration.api.internal.component.impl.MigrationApiImpl.performImport(MigrationApiImpl.java:460)
at com.collibra.dgc.migration.api.internal.component.impl.MigrationApiImpl.lambda$null$1(MigrationApiImpl.java:264)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at com.collibra.dgc.core.exceptions.ExceptionResolvingWithCachingTransactionTemplate.execute(ExceptionResolvingWithCachingTransactionTemplate.java:28)
at com.collibra.dgc.migration.api.internal.component.impl.MigrationApiImpl.lambda$performImport$2(MigrationApiImpl.java:254)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.collibra.common.exception.CollibraIllegalArgumentException: workflowConfigurationVariableUnknown
Params: [usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType]
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.validateThatNoUnknownVariableIsProvided(WorkflowServiceImpl.java:1750)
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.updateConfigurationVariables(WorkflowServiceImpl.java:1710)
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.updateConfigurationVariables(WorkflowServiceImpl.java:1611)
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.changeWorkflowDefinition(WorkflowServiceImpl.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy304.changeWorkflowDefinition(Unknown Source)
at com.collibra.dgc.core.api.component.workflow.WorkflowDefinitionApiImpl.changeWorkflowDefinition(WorkflowDefinitionApiImpl.java:105)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at com.collibra.dgc.core.aspect.ComponentCallLogAspect.logComponentCallInfo(ComponentCallLogAspect.java:78)
at sun.reflect.GeneratedMethodAccessor800.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at com.collibra.dgc.core.transaction.DGCApiV2Aspect.processDGCTransactionalAnnotation(DGCApiV2Aspect.java:45)
at sun.reflect.GeneratedMethodAccessor839.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at com.collibra.dgc.core.api.aspect.ApiExceptionHandlingAspect.mapExceptions(ApiExceptionHandlingAspect.java:33)
… 47 common frames omitted
com.collibra.common.api.exception.ApiIllegalArgumentException: com.collibra.common.exception.CollibraIllegalArgumentException: workflowConfigurationVariableUnknown
Params: [usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType]
at com.collibra.dgc.core.api.ApiExceptionTranslationHelper.translateToApiException(ApiExceptionTranslationHelper.java:57)
at com.collibra.dgc.core.api.aspect.ApiExceptionHandlingAspect.mapExceptions(ApiExceptionHandlingAspect.java:35)
at sun.reflect.GeneratedMethodAccessor838.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62)
Caused by: com.collibra.common.exception.CollibraIllegalArgumentException: workflowConfigurationVariableUnknown
Params: [usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType]
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.validateThatNoUnknownVariableIsProvided(WorkflowServiceImpl.java:1750)
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.updateConfigurationVariables(WorkflowServiceImpl.java:1710)
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.updateConfigurationVariables(WorkflowServiceImpl.java:1611)
at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.changeWorkflowDefinition(WorkflowServiceImpl.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)

43 Messages

Hi @francois.abraham.ext.orange.com, it seems that there is a conflict with workflow parameters. It’s quite difficult to say what could go wrong. Just a few ideas where to start:

  • Check if there is a workflow “OMRE - Description Update” deployed to your instance and remove it before the import process
  • try to import OMRE_v1.1.0_NoTableViews.cma instead of the OMRE_v1.1.0_Full.cma
    Hope it will help.

19 Messages

Hi, thanks for your answer.
Ther is no OMRE workflow deployed and i have the same issue with the OMRE_v1.1.0_NoTableViews.cma file instead of OMRE_v1.1.0_Full.cma.
Can you please identify what is the workflow that needs these params : usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType

It talks about “Escalation” (issue ?)
Any idea ?
Regards

43 Messages

Hi @francois.abraham.ext.orange.com, there is a workflow “OMRE - Description Update” with the parameters you mentioned.
I have attached the original workflow file here. Try to deploy it before the .cma import. OMRE-DescriptionUpdate.pdf (17.6 KB)
Since I couldn’t upload .bpmn file here I had to rename it to .pdf. So rename it back to .bpmn bore the deployment.

19 Messages

Hi, i have uploaded the OMRE-DescriptionUpdate.bpmn file and the new workflow is displayed . But i still have the same error when i try to import OMRE_v1.1.0_Full.cma file

Caused by: com.collibra.common.exception.CollibraIllegalArgumentException: workflowConfigurationVariableUnknown

Params: [usertask_proposeChangesEscalationType,usertask_checkProposedChangesEscalationDuration,usertask_requestRejectedEscalationDuration,usertask_checkProposedChangesEscalationType,usertask_proposeChangesEscalationDuration,usertask_requestRejectedEscalationType]

at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.validateThatNoUnknownVariableIsProvided(WorkflowServiceImpl.java:1920)

at com.collibra.dgc.core.workflow.impl.WorkflowServiceImpl.updateConfigurationVariables(WorkflowServiceImpl.java:1904)

43 Messages

Hi @francois.abraham.ext.orange.com, it doesn’t seem like an issue with OMRE itself. There is a problem during the .cma file import which was tested many times and so far it worked well.
I’d recommend to talk with Collibra Support about possible reasons of errors in the log-file.

2 Messages

3 years ago

Hi @lugovyi.dmytro,

Is this compatible with version 2021.06.3 or do you not advise it?
I only ask because the documentation mentions support for 5.7.x; 2020.10; 2020.11, while the marketplace page mentions any version newer than 5.7.

This seems like a great addition to Collibra and I intend on introducing it to the Prod environment as soon as possible.

Many thanks,
Faustino

43 Messages

hi @faustinogaskin.kubrickgroup.com, to be honest I’ve never tried it on the 2021.x but since there were no major changes in the Collibra Operating Model since 2020.x I’d assume that it should work.

2 Messages

Thanks, I’ll let you know how it goes.

2 Messages

3 years ago

Hi, when running OMRE 1.2 the workflow fails. Collibra displays Activity Failed errors e.g. ‘Code’ doesn’t have UUID, 'Dataset Technology ’ doesn’t have UUID etc.

I checked with Collibra support and the logs provided this example error:

2021-08-12 10:45:29.218 [jobExecutor-5] INFO c.c.d.c.api.component.LoggerApiImpl - [OMRE] Attribute Type ‘Code [0517c20b-0213-4dca-b5fc-6343f14c51cd]’ has empty UUID attribute [ehcache-txid=7298614, trace_id=26fb5c01ecdc0381ce20414c9ce474f1, trace_flags=01, span_id=d52c486279f3dc8e]
2021-08-12 10:45:29.318 [jobExecutor-5] WARN o.f.e.i.b.b.ScriptTaskActivityBehavior - Exception while executing scripttask_omre : problem evaluating script: javax.script.ScriptException: com.collibra.dgc.core.exceptions.DGCException: Code doesn’t have UUID [ehcache-txid=7298614, trace_id=26fb5c01ecdc0381ce20414c9ce474f1, trace_flags=01, span_id=55419d2bb51c5f32]

I have deleted some of these specific OMRE attribute types but the workflow still fails…and sometimes does not provide failure details.

According to the logs (provided by Collibra support) this ,appears to be the query that is running:

omreDomainIds.minus(validationRulesDomainId).each { domainId ->
assetApi.findAssets(FindAssetsRequest.builder()
.domainId(domainId)
.typeIds(omreAssetTypeIds.minus([permissionAssetId, startEventAssetId]).minus(assetValidationRuleUUID))
.limit(10000)
.build()
).getResults().each {
foundAsset ->
uuidValue = attributeApi.findAttributes(FindAttributesRequest.builder()
.assetId(foundAsset.getId())
.typeIds([attributeTypeUUID])
.build()
).getResults()
if (!uuidValue) {
loggerApi.info(workflowIdent + " " + foundAsset.getType().getName() + " ‘" + foundAsset.getName() + " [" + foundAsset.getId() + "]’ has empty UUID attribute")
def assetUrl = applicationApi.getInfo().getBaseUrl() + “asset/” + uuid2String(foundAsset.getId())
def assetUrlHtml = ‘<a href="’ + assetUrl + ‘">’ + assetUrl + ‘</a>’

def dgcError = new DGCException("${foundAsset.getName()} doesn’t have UUID ")
dgcError.setTitleCode(“Asset ${assetUrlHtml} has empty UUID attribute”)
throw dgcError
}

I can see via the release notes this is a known bug during the OMRE extraction process. Is there a recommendation on how to fix this issue? Should i delete the Operating Model community and re-run the workflow from the start? Any advice would be much appreciated. Thanks in advance. Simon

43 Messages

3 years ago

@simon.knee every asset extracted by OMRE should have UUID. Without UUID OMRE will not be able uniquely identify object and update it. If you have objects with empty UUID then OMRE will fail during the extraction. There are few options why it could happen:

  • someone created assets manually without adding UUID
  • someone has removed UUID
  • OMRE didn’t add UUID during the extraction
    In order to find the real reason please find the asset with empty UUID and check the Activity tab so to check what had happen.
    If there are custom assets with empty UUID - remove them and restart the OMRE
    If someone deleted UUID from the assets - you can add them back or remove assets and restart the OMRE
    If OMRE created asset without UUID - please redeploy OMRE 1.2 with workflows and restart it
    Let me know if it help

2 Messages

Hi Dmytro,

Thanks very much for your quick response.

As I couldn’t track all the assets with broken UUIDs and the worflow kept failing, I decided to redeploy OMRE 1.2. This required initially deleting the operating model community first and starting afresh. The workflow the ran successfully. We are now back in business. Thank you very much for your support.

Kind regards
Simon

77 Messages

 • 

250 Points

3 years ago

This tool is fantastic! Is it possible for OMRE to export all of the asset types and their related attributes? I have a need to create a data model in a different tool,and would like to avoid having to manually create all of the asset<>attribute relationships. I see how to export asset<>relationships.

Thanks,
Scott

43 Messages

@scott.benson I’m happy to hear that this tool helps you in your data governance activities. Regarding your question, after extraction of the OM objects, you can go to the Asset Types domain and extract them with all the characteristics. I’d also recommend you consider using Collibra for the representation of the data model. In some cases it can be very useful.
What you can do:

  • create a new asset with the type “Data Model”
  • relate this asset to those asset types which are included in this model
  • configure new traceability diagram that would show you the Data Model and included asset types with relations
    This is a screenshot of what you can get:


77 Messages

 • 

250 Points

3 years ago

@lugovyi.dmytro- That solves half of the question, very nicely. Now to get the individual attributes for each asset type, I have to link to Assignment, and from there to “Assignment Characteristics” and then explode each characteristic to get the individual attributes?

I was hoping there was an easier way - but I like it so far

I have a second question - “How do I re-run OMRE?” I have made some changes to the the Operating Model, and need them to be reflected in the OMRE diagram

Scott

77 Messages

 • 

250 Points

3 years ago

@lugovyi.dmytro - The first time I ran OMRE, it was perfect. Now, whenI tried to re-run it, I receive a workflow error. I need to re- run it because I made some changes to the model.

Thanks.

Scott

43 Messages

@scott.benson What is the error message? What changes you’ve made?

77 Messages

 • 

250 Points

@lugovyi.dmytro , once the wheel in the upper right corner stops spinning, it just says “failed”. No explanation

43 Messages

@scott.benson could you please check the log-file in the console? the error message should be there

77 Messages

 • 

250 Points

3 years ago

@lugovyi.dmytro - Sorry about the long delay - I didnt see your response. Today when I ran it, I recevied the error message “Activity Failed - Attribute Type has not been found”.

The log makes reference to "@FormParam will work as expected. Resource methods consuming the request body by other means will not work as expected.

Thanks

43 Messages

@scott.benson before the actual extraction of the OM objects OMRE checks all the necessary attribute types, asset types etc. In case if one of them missing OMRE will return the error you see. Check the log files for more details to see which attribute type is missed. Probably someone changed the name of the attribute type or removed it or OMRE has been installed incorrectly.

77 Messages

 • 

250 Points

3 years ago

@lugovyi.dmytro. ERROR c.c.d.c.s.i. AuthorizationServiceImpl ‘edge is not active or invalid’

7 Messages

2 years ago

Please note that we have observed, that as of Collibra DGC release 2021.04, if hyperlinks were enabled within DGC there have been issues running OMRE 1.2.0.
The issue is not in the OMRE 1.2.0 release but a bug which appeared in 2021.04. This bug doesn’t allow complex relation attribute creation if the text value in the attribute is recognised as hyperlink and ends up throwing a DataFetchingException.
This has been fixed as of version 2021.11 and OMRE is able to run without any issues.

38 Messages

@guillaume.le.galiard & [email protected]. Please see here over the latest updates for the OMRE problem, which is in fact a DG problem.

1.2K Messages

Can that explain why OMRE was running for over 12h on an instance of ~142k assets?

7 Messages

Hi @arthur.burkhardt, Please note that OMRE scans through all the assets, attributes, domains, communities, assignments, etc. The performance really depends on the amount of such resources in the Collibra instance. Although please note, that only the first run takes time, subsequent runs are faster as it only captures delta from the previous state. Hence it’s recommended to carry out the first run over a weekend.

2 Messages

6 months ago

Hello Experts,

We have an encountered unexpected error while running OMRE - Extraction and report workflow.

And I need your help to examine below warning and error message.

023-11-06 04:00:17.786 [jobExecutor-4] INFO c.c.d.c.api.component.LoggerApiImpl - [OMRE] 62 Domains found [session_hash=06d0d6e2cac09cb8cb0f3ae46a9e9959, authenticated_id=00000000-0000-0000-0000-000000900003, trace_id=2b7d2c3c9dfe67a3174c766eba139d14, trace_flags=01, span_id=edbd8be47157e776]
2023-11-06 04:00:17.794 [jobExecutor-4] INFO c.c.d.c.api.component.LoggerApiImpl - [OMRE] 23 Status Types found [session_hash=06d0d6e2cac09cb8cb0f3ae46a9e9959, authenticated_id=00000000-0000-0000-0000-000000900003, trace_id=2b7d2c3c9dfe67a3174c766eba139d14, trace_flags=01, span_id=9316e0d12cac0489]
2023-11-06 04:00:17.809 [jobExecutor-4] INFO c.c.d.c.api.component.LoggerApiImpl - [OMRE] 24 Domain Types found [session_hash=06d0d6e2cac09cb8cb0f3ae46a9e9959, authenticated_id=00000000-0000-0000-0000-000000900003, trace_id=2b7d2c3c9dfe67a3174c766eba139d14, trace_flags=01, span_id=fd2fc406ccba3acc]
2023-11-06 04:00:17.843 [jobExecutor-4] WARN c.c.d.w.s.g.b.SecureGroovyTaskActivityBehavior - Exception while executing scripttask_omre : groovy script evaluation failed: ‘javax.script.ScriptException: java.lang.UnsupportedOperationException’ Trace: scopeType=bpmn, scopeDefinitionKey=scriptOperatingModelReverseEngineering, scopeDefinitionId=17abe465-ba24-4814-ad36-b8a1c17c3382, subScopeDefinitionKey=scripttask_omre, tenantId=, type=scriptTask [session_hash=06d0d6e2cac09cb8cb0f3ae46a9e9959, authenticated_id=00000000-0000-0000-0000-000000900003, trace_id=2b7d2c3c9dfe67a3174c766eba139d14, trace_flags=01, span_id=759ebd09870add7a]
2023-11-06 04:00:17.849 [jobExecutor-4] ERROR c.c.d.c.w.a.WorkflowExceptionHandler - Error starting workflow with id: scriptOperatingModelReverseEngineering [session_hash=06d0d6e2cac09cb8cb0f3ae46a9e9959, authenticated_id=00000000-0000-0000-0000-000000900003, trace_id=2b7d2c3c9dfe67a3174c766eba139d14, trace_flags=01, span_id=759ebd09870add7a]
org.flowable.common.engine.impl.scripting.FlowableScriptEvaluationException: groovy script evaluation failed: ‘javax.script.ScriptException: java.lang.UnsupportedOperationException’ Trace: scopeType=bpmn, scopeDefinitionKey=scriptOperatingModelReverseEngineering, scopeDefinitionId=17abe465-ba24-4814-ad36-b8a1c17c3382, subScopeDefinitionKey=scripttask_omre, tenantId=, type=scriptTask
at org.flowable.common.engine.impl.scripting.ScriptingEngines.evaluate(ScriptingEngines.java:132)
at org.flowable.common.engine.impl.scripting.ScriptingEngines.evaluate(ScriptingEngines.java:105)
at org.flowable.common.engine.impl.scripting.ScriptingEngines.evaluate(ScriptingEngines.java:78)
at com.collibra.dgc.workflow.security.groovy.behavior.SecureGroovyTaskActivityBehavior.execute(SecureGroovyTaskActivityBehavior.java:77)
at org.flowable.engine.impl.agenda.ContinueProcessOperation.executeActivityBehavior(ContinueProcessOperation.java:298)
at org.flowable.engine.impl.agenda.ContinueProcessOperation.executeSynchronous(ContinueProcessOperation.java:175)
at org.flowable.engine.impl.agenda.ContinueProcessOperation.continueThroughFlowNode(ContinueProcessOperation.java:125)
at org.flowable.engine.impl.agenda.ContinueProcessOperation.run(ContinueProcessOperation.java:88)
at org.flowable.common.engine.impl.AbstractEngineConfiguration.lambda$new$0(AbstractEngineConfiguration.java:195)
at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperation(CommandInvoker.java:130)
at org.flowable.engine.impl.interceptor.CommandInvoker.executeOperations(CommandInvoker.java:114)
at org.flowable.engine.impl.interceptor.CommandInvoker.execute(CommandInvoker.java:72)
at org.flowable.engine.impl.interceptor.BpmnOverrideContextInterceptor.execute(BpmnOverrideContextInterceptor.java:26)
at org.flowable.common.engine.impl.interceptor.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53)
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:105)
at org.flowable.common.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:52)
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
at com.collibra.dgc.core.workflow.activiti.extensions.CustomCommandInterceptor.execute(CustomCommandInterceptor.java:24)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)
at com.collibra.dgc.core.workflow.activiti.WorkflowEngineImpl.startProcessWithFormProperties(WorkflowEngineImpl.java:440)
at com.collibra.dgc.core.workflow.activiti.WorkflowEngineImpl.lambda$startProcess$3(WorkflowEngineImpl.java:462)
at com.collibra.dgc.core.workflow.activiti.WorkflowEngineImpl.lambda$startProcessTemplate$5(WorkflowEngineImpl.java:487)
at com.collibra.dgc.core.workflow.activiti.WorkflowExceptionHandler$Executor.execute(WorkflowExceptionHandler.java:169)
at com.collibra.dgc.core.workflow.activiti.WorkflowEngineImpl.startProcessTemplate(WorkflowEngineImpl.java:485)
at com.collibra.dgc.core.workflow.activiti.WorkflowEngineImpl.startProcess(WorkflowEngineImpl.java:458)
at com.collibra.dgc.core.workflow.impl.WorkflowActionServiceImpl.startWorkflow(WorkflowActionServiceImpl.java:503)
at com.collibra.dgc.core.workflow.impl.WorkflowActionServiceImpl.startWorkflow(WorkflowActionServiceImpl.java:163)
at com.collibra.dgc.core.workflow.impl.job.WorkflowJobRunner.startWorkflowForItem(WorkflowJobRunner.java:55)
at com.collibra.dgc.core.workflow.impl.job.AbstractWorkflowJobRunner.startForBusinessItems(AbstractWorkflowJobRunner.java:75)
at com.collibra.dgc.core.workflow.impl.job.AbstractWorkflowJobRunner.run(AbstractWorkflowJobRunner.java:57)
at com.collibra.dgc.core.workflow.impl.job.WorkflowJobRunner.run(WorkflowJobRunner.java:24)
at com.collibra.job.api.IJobRunner.execute(IJobRunner.java:53)
at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob$1.doInTransactionWithoutResult(JobServiceImpl.java:1207)
at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:36)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
at com.collibra.dgc.core.exceptions.ExceptionResolvingWithCachingTransactionTemplate.execute(ExceptionResolvingWithCachingTransactionTemplate.java:28)
at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.runJobInTransaction(JobServiceImpl.java:1201)
at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.postInitRun(JobServiceImpl.java:1106)
at com.collibra.dgc.core.service.job.impl.JobServiceImpl$RunJob.run(JobServiceImpl.java:1068)
at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:89)
at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:152)
at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:94)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: javax.script.ScriptException: javax.script.ScriptException: java.lang.UnsupportedOperationException
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:158)
at java.scripting/javax.script.AbstractScriptEngine.eval(Unknown Source)
at org.flowable.common.engine.impl.scripting.ScriptingEngines.evaluate(ScriptingEngines.java:111)
… 45 common frames omitted
Caused by: javax.script.ScriptException: java.lang.UnsupportedOperationException
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:320)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:155)
… 47 common frames omitted
Caused by: java.lang.UnsupportedOperationException: null
at java.base/java.util.ImmutableCollections.uoe(Unknown Source)
at java.base/java.util.ImmutableCollections$AbstractImmutableList.sort(Unknown Source)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.sort(DefaultGroovyMethods.java:9887)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.sort(DefaultGroovyMethods.java:9851)
at org.codehaus.groovy.runtime.dgm$683.invoke(Unknown Source)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:242)
at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:51)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
at Script6.run(Script6.groovy:545)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:317)
… 48 common frames omitted

Thanks, in advance.
Anirudh Goel

43 Messages

Hi @anirudh.goel.underarmour.com, it seems that the error happens right at the moment of the Scopes extraction. So question - how many Scops do you have configured in Collibra?

2 Messages

Hi Dmytro,

We have zero scopes as of now and OMRE extraction workflow was working till last week with zero scopes.
Regards,
Anirudh Goel

8 Messages

We got the same error message. I haven’t looked into it completely yet, but it’s indeed with the scopes. By the looks something that has to do with the sorting or the list of scopes. If you remove that part of the workflow (OMRE - Extraction and Report - somewhere around line 530) it probably works again. Somehow the sorting causes the unsupported operation exception.

Edit: coworker also took a look. The api currently returns AbstractImmutableList which doesn’t have a sort method. Easiest fix is to cast it, such as

def allScopes = (new ArrayList(scopeApi.getAllScopes())).sort { it.getLastModifiedOn() }

and you should be good to go

7 Messages

5 months ago

Hi, As per our internal discussions, there was a change made to ScopApi.getAllScopes() method now returns .stream().map(…).toList() which is an immutable list.

We will try to make a new fix available soon on marketplace.

70 Messages

3 months ago

Hi @roshan.medatwal.collibra.com @lugovyi.dmytro,

When can we expect to have the updated version of code . Scope Api is giving an error, and prohibits the workflow from running.

Regards,
Ravi

13 Messages

 • 

800 Points

Hi Ravi,

Please go to the OMRE Extraction Report Groovy code and replace
def allScopes = scopeApi.getAllScopes().sort { it.getLastModifiedOn() }
with this
def allScopes = (new ArrayList(scopeApi.getAllScopes())).sort { it.getLastModifiedOn() }

It will work

Thanks,
Paresh

70 Messages

Scope error is fixed, but the workflow still doesn’t work. Still facing error: workflow says unable to get first() element. There are more than 5000 lines of code, it would be helpful if Collibra fixes this workflow asap. It’s very useful.

@roshan.medatwal.collibra.com @lugovyi.dmytro @bineesh.babu.collibra.com

3 months ago

I would also like to see the OMRE workflow fixed by Collibra for those of us who are not developers.

Loading...