Integrating CloudCompare with Unity for AR Development

[meshdifferencing]

Hello everyone.

I'm trying to find a way to 3D scan the environment, and then immediately compare this newly generated 3D mesh with the pre-loaded digital twin of the environment. For example, in the digital twin captured yesterday, a bookshelf has 10 books on it. In the present, the user is looking at the bookshelf with their augmented reality device, and there is now a helmet on the shelf, and 5 books are missing. I want the augmented reality application to display the added features (helmet) in green, and missing features (5 books) in red.

I've successfully been able to compare two different 3D meshes of the same object using CloudCompare, I'm stuck on how to move forward with Unity.

Does anyone have experience with integrating CloudCompare's functionality of comparing the differences between two 3D meshes with Unity?

Thanks in advance for your help

[daniel]

Not on my side (I did that with an ancestor of Unity which was Virtools, but they had a C++ framework, while I believe Unity is more C# isn't it?).

Maybe others have more info / experience.

[meshdifferencing]

Hi Daniel,

Could you elaborate a bit more on your experience in doing this in Virtools?

Unity is indeed C#. In order to do what I need to do in CloudCompare, I use the 'Align two clouds by picking at least 4 equivalent point pairs' tool and the 'Compute cloud/mesh distance' tool. I would greatly appreciate if you could point me in the right direction as to how implement these two functionalities in C# as a Unity application. For example, which source files are necessary for these two tools to run?

Thanks again for your help

[daniel]

No that's actually the issue with C#/Unity.

With Virtools, you had the option to create a C++ "plugin" that was actually making the interface between the Virtools SDK and CCLib. This plugin was responsible for converting data structures from Virtools to CC, then call the CCLib methods, and then convert the structures back to Virtools.

With Unity, this "plugin" must be coded in C# and then I don't know how one can call CCLib which is a C++ library. The only way I know is "marshalling" but it's painful, and in this case I'm not even sure it's possible since the structures that needs to be exchanged (clouds, etc.) are very complex.