Spring boot Integration to AWS Lake

Hi @vani.nithya.indla ,

Currently, the AWS integration is responsible for any items in Lake Formation, assuming that the right permissions are set. Can you give us a bit more details on such partition columns, please?
Thanks

Hi @vani.nithya.indla,

We are still investigating the partitions column issue.

Regarding the second reply, the log screenshot shows the issue happening when trying to check the Collibra version.

Firstly, have you tried running the integration more than once since there could be something temporarily wrong with the internet connection?

If the issue persists, you can try upgrading the integration library version to version 1.1.9 by following the steps in the documentation under the ‘Updating the Library Jar File’ section.

For us to investigate further, we would like to know the following:

• What version of Collibra are you currently running?

• Can you send us a private message of the integration library logs?

You can do this by adding the following log to the src>main>resources>application.properties file;
collibra.log.http-requests-at-info=true
However, please remove any sensitive information from these logs, such as usernames and passwords.

Thanks

Hi @spring-team.collibra.com

Thanks for your response.

We have identified that issue is caused by a network error and team is in contact with collibra support to get the details. And the partition key issue still exists and need investigation.

A query regarding .cma file in latest version
we could find two .cma files in latest package provided by Collibra

Can you please confirm which one to upload, we found that there is no difference between AWS_Lake_formation.cma shared in V.0.1 release to v.1.0.3 but it is mentioned as .cma updated in latest release notes

Hi @vani.nithya.indla ,

The correct CMA file is 'sbiLakeFormation.cma’. Please note that for the CMA file to use the correct IDs, remove all custom assets/attributes/relations from the previously imported CMA file. If you face any errors, you can check the custom constants file in src/main/java/com/collibra/aws/lakeformation/ util / location and make sure that the IDs match the ones specified on the Collibra Data Intelligence Cloud instance.

Thanks

Hi @spring-team.collibra.com

we have two environments dev and pre prod , and when we migrate data from one environment to other using .cma file, attribute type UUID’s are different in both the environments. Can you confirm if we can retain the same UUID’s for attribute types in both the environments

Hi @vani.nithya.indla,

In the latest version of the CMA file, there were changes in certain assets’ IDs.

Therefore, if the integration is working on the current dev instance and you want the IDs to match between both instances, we suggest building the CMA file directly from your dev instance containing the custom assets/attributes/relations used by the AWS integration. Then import it to your pre-prod instance.

Thanks

Hi @spring-team.collibra.com

Is there any documentation on relationships between AWS lake formation asset types and attributes?

Hi @vani.nithya.indla ,

Yes, the following sections in the marketplace documentation

contain information about the asset and attribute types;

• Collibra Design And Metamodel
• Asset Types
• Attribute Types
• Relation Types

However, should you need any more information, please let us know.

Thanks

Hi @vani.nithya.indla,

Thanks for the suggestion! We’ll add a section relating to the asset types and attributes in the documentation. In the meantime, however, you can download the excel file from the Collibra instance right before you import the CMA file by clicking on the ‘Download Simulation’ option, as seen from the screenshot.

Thanks