Segment anything

[PablerasBCN]

segment anything implementation idea.

May be we could rasterize current view, run segment anything on it, then project the colors into the cloud.

I know the color projecting may not be easy but has been a forever requested feature, so here another excuse to implement it :D

this is a segmentation from a screen capture of a point cloud



to make it perfect, may be the ideal scenariow ould be to move camera over a path and every nth frames segment anything, this way if a point accumulates a class n number of times you would get a more robbust segmentation (I think).

Just an idea. Cheers.

[daniel]

Does "segment anything" means something in particular? Or is it a manual process? (sorry, I don't understand)

[PablerasBCN]

Apologies!

I thought this would be more known. Not too long ago facebook released a segmentation model that seems to work quite nicely. I tested on an screen capture of a point cloud and certainly idientifies objects properly.

Segment Anything
https://github.com/facebookresearch/segment-anything

So simply as an small chat idea said that it would be cool to have the hability to screen capture, or render current view at XY resolutions, run the model and classify points in viewport based on the segmentation by projecting the segmented image over the cloud as SF.

[daniel]

Ah, interesting. Sadly it's a Python repository. I won't be able to do anything with it myself. But maybe via CloudCompy or the Python plugin, someone skilled enough could come up with something indeed.

[SD_cc]

I've been testing Segment Anything on rasterized images of point clouds. the results are super encouraging! In my particular case, the resulting polygon is used to crop the point cloud (using CC CLI). Could be interesting to transpose into a plugin...

[jason3d]

Ive been following looking for pointcloud implementations, saw a bunch of GIS projects adding it or something similar, also came across https://github.com/Pointcept/SegmentAnything3D