Talend ETL to Collibra Integration

This Spring Boot integration parses Talend ETL metadata, transforms, and upserts it to a Collibra Platform instance as assets.
The supported connection types are:
TalendMetadata:XmlFileConnection
TalendMetadata:DelimitedFileConnection
TalendMetadata:DatabaseConnection

When I tried to test with the delimited connection file(.dat), Integration throwing the following error while parsing the file content.

2022-05-04 00:03:37,382 [https-jsse-nio-8442-exec-10] INFO com.collibra.marketplace.talend.etl.service.TalendEtlFileService - Processing Talend ETL file content.
2022-05-04 00:03:37,385 [https-jsse-nio-8442-exec-10] ERROR com.collibra.marketplace.talend.etl.exception.GlobalExceptionHandler - Internal error during execution.
com.collibra.marketplace.talend.etl.exception.TalendEtlException: An error occurred while retrieing the data. Details: Error on line 1 of document : Content is not allowed in prolog.

Can you please help me on this.
Thanks,
Supriya

Hello @spring-team.collibra.com,
Could you kindly send us a sample Talend ETL file for retrieving metadata as a Delimited File Connection?
This integration parses metadata from a Talend ETL file and imports it as assets into the Collibra Platform.
I tested with a sample dat file what we have, the spring boot integration fails to parse the ETL metadata file.

Thanks,
Supriya

Hello Supriya,

Sure, attached please find a sample .item file containing a DelimitedFileConnection.
(Please rename the file from .txt. to .item once downlaoded)

sample-delimited-file.item (2.5 KB)

@supriya.abbineni.kofc.org Apologies I misspoke earlier - This is a community offering so support teams are not enabled to give support to it unfortunately. With that said, thank you @spring-team.collibra.com for sending along that file.

Hello Supriya,

To identify the attribute types’ resource ID, the following may be done:

1. Go the Collibra instance, Settings page
2. From the Operating Model section, select Attributes
3. On the right-hand side, there should be the following icons:
   
   
4. Select the one marked with a red box
5. Then, select Columns and check Resource Id:
   
   
   
   
6. Once done, a new column Resource Id would be added. This is the one that should be used.

Hello Supriya,

Regarding the latest error encountered, can you please try create a scope as specified here.

This scope should be assigned to the Collibra domain that is being used by this integration.

Subsequently, can you please for each asset type defined in the documentation Collibra Metamodel section assign the created scope to the asset type and then add the custom assignments (Assignments section from the documentation) to this scope.

@spring-team.collibra.com Is this something you could help @shanthi.subramanian.transport.nsw.gov.au with?

Hello Shanthi,

Thanks for trying the Talend ETL to Collibra integration.

Currently, it does not include a CMA file. Will keep you updated once is has been added and is available on Collibra’s Marketplace.

Thanks Kristen and spring Boot Team

Hello Shanthi,

Just to let you know that v1.0.6 of the Talend ETL to Collibra integration which includes the CMA file has been released:

Hi Team,

I am in the process of doing talend etl to collibra integration. For encryption, I downloaded the jasypt-1.9.3 jar file in my local directory. I followed the steps provided in the documentation. When i execute through command line,i am receiving “Could not find or load main class org.jasypt.intf.cli.” error.Any help/advice would be really appreciated.

Thanks,

Thanks all. I solved this issue.

Hello Ryan,

Thanks for your interest in the Talend ETL to Collibra integration.

This integration consumes the .item files and should be compatible with Talend Open Studio v7.

Meanwhile, regarding Talend Cloud v8, it was not designed/tested against .item files generated using this version and therefore, it might not work as expected.

Hi,

I am trying to build the project using ECLIPSE IDE.But i received the below error.I updated the version element in pom xml file.Can anyone help me to solve this?

Thanks

[WARNING] Some problems were encountered while building the effective model for org.flowable.examples:flowable-examples:jar:1.0-SNAPSHOT
[WARNING] ‘build.plugins.plugin.version’ for org.apache.maven.plugins:maven-eclipse-plugin is missing. @ line 60, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] Downloading from : https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml
[INFO] Downloading from : https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml
[INFO] Downloaded from : https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (14 kB at 8.6 kB/s)
[INFO] Downloaded from : https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (21 kB at 12 kB/s)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.230 s
[INFO] Finished at: 2022-07-06T15:08:00+10:00
[INFO] ------------------------------------------------------------------------
[ERROR] No plugin found for prefix ‘spring-boot’ in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException

@spring-team.collibra.com Is this something you could help @shanthi.subramanian.transport.nsw.gov.au with or direct someone else here that could?