Talend ETL to Collibra Integration

@shanthi.subramanian.transport.nsw.gov.au Also, does this other “TalendETL” thread help at all? - https://community.collibra.com/articles/knowledge-base/the-page-you-are-looking-for-no-longer-exists/6638e69271654016712feda4

Hi @shanthi.subramanian.transport.nsw.gov.au ,

Since you are using eclipse to run the integration, did you follow the steps to set up the run configurations settings under the Spring Boot Integration - Development section?
Can you please from the terminal/command prompt do the following:

1. Go to the integration root folder (i.e. where the pom.xml file is located)
2. Execute the following command to deploy the integration: mvn spring-boot:run
3. Check whether the integration is deployed successfully

Thanks

Hi ,i received the below error

Hi @shanthi.subramanian.transport.nsw.gov.au,

Can you please confirm what changes were made to the pom.xml file?

Additionally, would it be possible to try downloading the latest version from Collibra’s Marketplace and then after setting the required properties (in the application.properties file), try to deploy the integration using Command Prompt?

Hi,I downloaded the latest version .updated the properties file.I deployed using command prompt using this command "mvn spring-boot:run .I received the below error.Any help would be really appreciated.Thanks
Caused by: java.lang.IllegalArgumentException: Password cannot be set empty
at org.jasypt.commons.CommonUtils.validateIsTrue(CommonUtils.java:150) ~[jasypt-1.9.3.jar:?]
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.resolveConfigurationPassword(StandardPBEByteEncryptor.java:851) ~[jasypt-1.9.3.jar:?]
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.cloneAndInitializeEncryptor(StandardPBEByteEncryptor.java:528) ~[jasypt-1.9.3.jar:?]
at org.jasypt.encryption.pbe.StandardPBEStringEncryptor.cloneAndInitializeEncryptor(StandardPBEStringEncryptor.java:483) ~[jasypt-1.9.3.jar:?]
at org.jasypt.encryption.pbe.PooledPBEStringEncryptor.initialize(PooledPBEStringEncryptor.java:405) ~[jasypt-1.9.3.jar:?]
at org.jasypt.encryption.pbe.PooledPBEStringEncryptor.decrypt(PooledPBEStringEncryptor.java:502) ~[jasypt-1.9.3.jar:?]
at com.ulisesbocchio.jasyptspringboot.encryptor.DefaultLazyEncryptor.decrypt(DefaultLazyEncryptor.java:114) ~[jasypt-spring-boot-3.0.0.jar:?]
at com.ulisesbocchio.jasyptspringboot.resolver.DefaultPropertyResolver.lambda$resolvePropertyValue$0(DefaultPropertyResolver.java:44) ~[jasypt-spring-boot-3.0.0.jar:?]
at java.util.Optional.map(Optional.java:260) ~[?:?]

Hello @shanthi.subramanian.transport.nsw.gov.au,

Thanks for the update.

Based on the following error, it seems that the trigger.api.password might not be set in the application.properties file.

Caused by: java.lang.IllegalArgumentException: Password cannot be set empty

Can you please confirm that from the src/main/resources/application.properties file, the following property has a value set (i.e. a secure password that can be encrypted as explained in the documentation)? Thanks

 trigger.api.password=

Just to note that there are other properties that are required as well. These include the following:

server.port
trigger.api.username
trigger.api.password
collibra.url
collibra.username
collibra.password
collibra.domain
collibra.community

Hello @shanthi.subramanian.transport.nsw.gov.au,

Thanks for the update.

This error appears to be due to the SSL KeyStore file not being found.

Can you please confirm that the path set to the server.ssl.key-store property is correct and since a Windows machine is being used it starts with the drive letter, example C:\ (as it seems that currently it is being prefixed with file:)?

Hello @shanthi.subramanian.transport.nsw.gov.au,

Regarding the alias error, can you please confirm that the alias of the certificate located in the KeyStore file used is the same value set to the following property:

 server.ssl.key-alias

Just to note that there are other properties that should be updated when SSL is enabled on the listening endpoint. These are:

• server.ssl.key-store -> The location of the KeyStore file
• server.ssl.key-store-password -> The password of the KeyStore file
• server.ssl.key-alias -> The key alias
• server.ssl.key-password -> The password of the key located in the KeyStore file

Hello @shanthi.subramanian.transport.nsw.gov.au,

Thanks for the update.

It could be that the following is not an error but just an information log indicating that the pool size would default to 1.

Encryptor config not found for property jasypt.encryptor.pool-size, using default value: 1

Can you please provide the full error message/stack trace? Thanks

Hi Team,
When i trigger the integration followed by the steps provided in the document,i received the status 400 bad request error. Can anyone help me to solve this?

Thanks

Hello @shanthi.subramanian.transport.nsw.gov.au,

Regarding the error encountered, can you please send the Talend ETL integrations log file as a private message to determine what might be causing the 400 error?

Additionally, since the HTTPs post request requires an item file (i.e. the file that should be processed), can you please also confirm that the request made has the following structure (as explained in the documentation)? Thanks

curl --location --request POST 'https://localhost:8442/sync' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: Basic <encoded-auth>' \
--form 'itemFile=@"/sample_talend_etl_connection_file.item"'

Sure .I will send .Thanks

Hello @shanthi.subramanian.transport.nsw.gov.au,

Thanks for the information provided.

From the logs, the following error, it seems that the password of the Collibra instance might not be set:

{“message”:“There is a problem authenticating with the Collibra Platform instance.”,“details”:“Could not authenticate with the specified Collibra Platform. Response status code is 400 Payload: {“statusCode”:400,“titleMessage”:“Value not allowed”,“userMessage”:”‘password’ cannot be null nor empty."}"}

Can you please confirm that the collibra.password property located in the application.properties file is set? Thanks

Hi,

When i trigger the integration using POSTMAN,i received below error,

Can you please guide me how to fix this?

when SSL is enabled in postman:

Error: self signed certificate

when SSL is disabled in postman:

{“message”:“Internal error during execution.”,“exceptionMessage”:"{“message”:“An unexpected error occurred during execution.”,“details”:“Error reading entity from input stream.”,“cause”:“Unrecognized field \“result\” (class com.collibra.internal.api.model.output.Job), not marked as ignorable (17 known properties: \“userId\”, \“createdOn\”, \“resourceType\”, \“endDate\”, \“name\”, \“system\”, \“startDate\”, \“state\”, \“visibility\”, \“cancelable\”, \“lastModifiedBy\”, \“type\”, \“id\”, \“lastModifiedOn\”, \“createdBy\”, \“progressPercentage\”, \“message\”])\n at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 376] (through reference chain: com.collibra.internal.api.model.output.Job[\“result\”])”}"}

Hello @shanthi.subramanian.transport.nsw.gov.au,

Thanks for the update.

Yes, if a self-signed certificate is used, the SSL verification can be disabled.

Regarding the error above, can you please try installing and using the latest version of the Collibra Integration Library having v1.1.8?

Once downloaded, the following would be required:

1. Install the library JAR files as documented in the “Installation” section;
2. Open the pom.xml of your Talend ETL integration and set the “collibra.integration.library.version” property to 1.1.8.

(These steps are also documented in section “Updating the Library JAR File”)